Evaluation capability building: practical approaches

Facilitators: Florent Gomez, Jade Maloney, George Argyrous, Vanessa Hood, Brad Astbury

During this session, presenters will first go through a few practical approaches to Evaluation capability building (coaching, external training, communities of practice), how they fit within a broader framework, practical examples, what worked and what they are particularly suited for. You will then have the opportunity to contribute to an interactive discussion based on your own experience with such or other approaches and asking questions to the panel.

Florent Gomez is an Evaluation Manager in the NSW Department of Customer Service where he launched a successful Evaluation Community of Practice which is about to celebrate its 4 years anniversary. Before joining the NSW Government, Florent was an external evaluator for 10 years, first at Ernst & Young, Paris office and then at ARTD Consultants. Florent has conducted numerous evaluation projects for organisations ranging from the European Commission to NGO and regulatory agencies in recent years. Florent chairs the NSW Committee of the Australian Evaluation Society and sits on the steering committee of the Australian Public Sector Evaluation Network. He is passionate about sharing his experience in the evaluation field – even better with a touch of French sociology!

Jade Maloney is a Partner and Managing Director at ARTD Consultants. She works with government agencies, not-for-profits and citizens to co-design, communicate and evaluate social policies, regulatory systems and programs. Jade is passionate about ensuring citizens have a voice in shaping the policies that affect their lives, translating research into real world contexts, and ensuring evaluations are useful and used. Her communications work draws on her experience as a creative writer, time in the publishing industry and design studies.

Dr. George Argyrous is a Research and Evaluation Manager at the UTS Institute for Public Policy and Governance (IPPG) and was formerly ANZSOG Senior Lecturer in Evidence-Based Decision-Making and director of the Evidence and Evaluation Hub. He is an expert in the application of statistics and research methods to social, economic and evaluation research. George has managed and taught a number of research methods, statistics and evaluation training courses for public sector agencies, including the ACT Government, the NSW Agency for Clinical Innovation and the WA Public Sector Commission. He has also worked as a consultant to many public and private sector agencies on a range of research projects and evaluations.

Vanessa Hood blends the world of facilitation and evaluation and has over 15 years' experience in a range of settings, including behaviour change for sustainability. Vanessa is passionate about working with people and uses a range of creative facilitation techniques to help participants engage deeply with technical content and, importantly, with each other. In her current role as Associate Director with Rooftop Social she works with a range of NGOs and government organisations across Australia, including the Department of the Environment and Energy in Canberra and Financial Counselling Australia in Melbourne. She regularly delivers structured training, coaching and mentoring in facilitation and evaluative thinking. Prior to this, Vanessa was the Evaluation Lead at Sustainability Victoria, where she had responsibility for numerous internal evaluations at project and strategic organisational levels. She lives in Brunswick West in inner Melbourne with her family, including four gorgeous children, supportive hubby and four cheeky chooks.

Dr Brad Astbury is a Director at ARTD Consultants. He has 20 years’ experience in evaluation and applied social research. He has managed and conducted needs assessments, process and impact studies and theory-driven evaluations across a wide range of policy areas for industry, government, community and not-for-profit clients. Prior to joining ARTD in 2018, Brad worked for over a decade at the University of Melbourne, where he taught and mentored postgraduate evaluation students.

AES Evaluator Competencies Self-Assessment Part 1: Updates!

Facilitators: Amy Gullickson, Taimur Siddiqi, Lauren Wildschut, Sarah Mason, and the AES Pathways Self-Assessment Working Group

Last year at FestEVAL we alpha tested the AES Evaluator Competencies Self-Assessment - thanks to all of you that participated! We're back this year with two sessions. In this first session we’ll discuss the benefits of self-assessment, to report on what we changed based on your feedback and give you time to try out the revised self-assessment in the Learn Evaluation Assessment Portal.

Amy Gullickson is the Director of the Centre for Program Evaluation at the University of Melbourne.

Taimur Siddiqi is an evaluation consultant and Managing Director of The Incus Group.

Lauren Wildschut directs the Evaluation Studies Programme at the Centre for Research on Evaluation, Science and Technology at Stellenbosch University

Sarah Mason is the Director of the Center for Research Evaluation at University of Mississippi

The AES Pathways Self-Assessment Working Group

Evaluation within the public sector

Facilitated by Josephine Norman, APSEN Chair; Dr. Christabelle Darcy, APSEN steering group member

The session will begin with a brief update from the Chair of the Australian Public Sector Evaluation Network (APSEN) on recent activities followed by a very interesting session from ANU researchers (one of Dr Bray or Professor Gray) on their research paper Evaluation and Learning from Failure and Success.

The session will be introduced by our steering committee member Dr Christabelle Darcy, Assistant Director Program Evaluation for the Northern Territory Government, who drew on their findings as she stood up the whole of NT evaluation approach.

Josephine Norman: Jo is a manager within the Victorian Department of Health Centre for Evaluation and Research Evidence, where she leads the delivery on high profile and sensitive evaluations and reviews, such as the Review of the Medically Supervised Injecting Room. Jo is a Criminologist by training, has conducted more than a thousand individual interviews with vulnerable adults and young people, and loves the opportunity to better understand causality through the extensive linked datasets within the public sector. She has been the Chair of the Australian Public Sector Evaluation Network for the past 18 months.

Dr. Christabelle Darcy: Christabelle leads the Program Evaluation Unit within the Northern Territory Department of Treasury and Finance and is co-convenor of the Northern Territory branch of the Australian Evaluation Society. Christabelle has a background in health research and has previously worked in science and innovation policy with the Commonwealth Government and economic policy with the Northern Territory Government.

Professor Matthew Gray is Director of the ANU Centre for Social Research and Methods, with previous appointments include Director of Research for the College of Arts and Social Sciences and Deputy Director of the Australian Institute of Family Studies. He has published research on a wide range of social and economic policy issues including those related to Indigenous Australians. Professor Gray has extensive experience in evaluating major government policies and programs including the areas of family law, place-based interventions, labour market programs, income management and the 2018 changes to the childcare system.

Dr Rob Bray joined the ANU after a long career in the Australian Public service. In policy and program areas including employment, health, aged care, housing and social security he worked at the interface of economic and social policy in a range of roles including program management, policy development and evaluation. At the ANU he has been involved in a number of major evaluations including of Income Management in the Northern Territory, and most recently the Child Care changes introduced in 2018.

Design and Evaluation for Impact: Annual Meeting of the D&E Special Interest Group

Facilitated by Matt Healey and Jess Dart

The Design and Evaluation SIG provides a space for evaluators to explore the interface between design and evaluation, and the systems within which we undertake these practices. For the purposes of this SIG, we define ‘design’ as “purposeful and systematic processes to create positive change”. These processes encompass human-centred and collaborative approaches, and cover complex problems within the social, health and environmental content areas.

In previous years, the SIG has identified three interpretations of the interface between design and evaluation:

• Evaluators as designers (stepping into design roles)

• Evaluators as collaborators (working as part of design teams)

• Evaluators using design (applying design tools to evaluation)

Drawing on these as a foundation for the session (and providing practical examples), we will briefly cover the SIG’s purpose and outline what the group has explored in the past. Then, like good evaluators and designers, we want to question some of our assumptions about the topics and areas of interest going forward. The SIG is now entering its fifth year of operation and we are excited to hear more from you about what you want to see (or not see). We welcome new people.

Matt Healey is a director at First Person Consulting. He has a strong interest in approaches to the design of projects, programs and services. This ranges from human-centred design, co-design, the use of evidence and working with clients to figure out 'what works' for them. In recent times this has extended to systems-based approaches to understanding complex problems, and what we might do to address these. Matt is Secretary of the Australian Evaluation Society (AES) Special Interest Group (SIG) on Design and Evaluation.

Dr Jess Dart is the founder of Clear Horizon Consulting. Receiving the 2018 “outstanding contribution to evaluation” award, and the 2006 “developments in evaluation” award, Jess is a recognised leader in evaluation and strategic planning with over 30 years’ experience. She specialises in the evaluation and design of complex initiatives. She is passionate about ensuring that evaluation leads to improved social and environmental justice.

FestEVAL Club

The FestEVAL Club is a relaxed place to drop in and chat with colleagues. Our facilitators and MCs will be there to manage the proces. The Club is strictly BYO – grog and bean bag – so settle in for as long or little as you like!

Tuesday 

“Twists and turns along the way”

An opportunity to meet other evaluators and discuss aspects of our professional journeys; from thinking about ‘How I got into evaluation’ to ‘Why I’m still here and hanging in there’.

Call for Workshops for AES Professional Development – themes, calls and selection

Presented by the Pathways Committee working group

This presentation will provide prospective workshop suppliers with insight into the selection of workshop themes, the criteria used to assess proposals and shortlisting. The Pathways Committee embarked on an extensive research process to identify the gaps in current workshops and thematic interest to extend the current range of workshops offered. This Q&A session will be an opportunity to consult with working group members about the process, where they see the workshop program moving to in the future, and proposal development.

Moving Past the Black Tick in Evaluation

Facilitated by Skye Trudgett

Some recent attempts to centre and value First Nations peoples within evaluation practice have unfortunately maintained an undercurrent of Western epistemological, ontological and methodological superiority. First Nations evaluation has been practiced since time immemorial, and in order to genuinely enact First Nations agency and self-determination in this space, the exclusive act of premising Western evaluation approaches must cease. There is an immediate need to give space, build confidence and amplifying the voice of First Nations peoples and communities in the evaluation sector. Promisingly, there is a strong focus on shifting mindsets, cultural safety and Indigenous Data Sovereignty to scaffold progress forward; this session seeks to provide a practical, evidence based framework for First Nations communities to define and implement principles of Indigneous Data Sovereignty (and subsequently elements cultural safety) within design, evaluation and research.

The presentation will elaborate on Skye's short presentation at the opening: It is not progress, it is not data sovereignty and it is not self-determination when 'they' ram down your throat patriarchal evaluation practice, then seek out your black tick.

Evaluators navigating ethics 

Presented by Lisette Kaleveld, Keryn Hassall, Keren Winterford, Racheal Laugery, Samantha Abbato, Eleanor Williams

This panel presentation will explore key questions in relation to evaluation and ethics: What do we mean when we talk about ethics in evaluation? What do we need to consider and how can we translate principles into practice?

The panel will explore these questions from different dimensions including: theory; principles; rules and guidelines; and navigating ethics in practice. This session will provide an opportunity to highlight a range of issues and seek input from attendees about common challenges and questions.

The purpose of the two seminars is to provoke critical thinking on the topic of ethics and provide space for practitioners to share their experiences and reflect on their practice.

Lisette Kaleveld is a social researcher based at the Centre for Social Impact (UWA), and has also worked for the public service, the not-for-profit sector and as an independent consultant. She has conducted policy reviews and evaluations in mental health, disability, homelessness, the arts and education sectors. Recently, Lisette has worked with more focus on bringing in co-design methodologies and working with lived experience expertise as an important dimension of knowledge and evidence, especially when dealing with complex social problems. Lisette has also worked as a journalist, has a Graduate Diploma in Journalism, a Bachelor of Science (Anthropology), and is Convener of the Western Australia Regional Network Committee of the Australian Evaluation Society.

Keryn Hassall is an evaluator working across many policy areas. She uses ideas from philosophy, public administration research and organisation theory to inform evaluation practice. Her consultancy work is primarily formative and process evaluation helping government agencies to clarify program theory, understand the context, improve program design and implementation, and build the craft skills of public administration to support program effectiveness.

Dr Keren Winterford has more than 20 years work experience in the international development sector, in multiple capacities with Managing Contractors, NGOs, as a private consultant, and more recently in development research. Keren's areas of expertise include research design in partnership, particularly with NGOs; participatory research; facilitation and training; design, monitoring and evaluation; and strengths-based approaches to development. She has a passion and commitment to ethical research practice. She has sectoral experience and expertise related to gender and social inclusion, citizen participation, governance and social accountability.

Racheal Laugery is the Acting Research Ethics Manager at the University of Technology Sydney (UTS) and has worked in the area of research ethics for over a decade in a variety of roles, covering human ethics, animal ethics and biosafety. She provides individualised expert advice to academics and HDR students, not just through the preparation and submission of ethics applications, but throughout the research lifecycle.

Dr Samantha Abbato is an experienced evaluator and trainer for the Australian Evaluation Society (AES). Sam has more than twenty-five years of experience and strong methodological expertise across a range of qualitative and quantitative disciplines. Her academic grounding in quantitative methods is built upon a Bachelor's degree in mathematics and physiology and a Masters of Public Health in epidemiology and biostatistics. She has extensive qualitative training in medical anthropology (Ph.D., UC Berkeley). As an independent consultant, Sam has assisted a range of clients, including non-government organisations and local, state, and Commonwealth governments. As the Director of Visual Insights People since 2013, she has introduced a pictures and stories approach to evaluation for the health and community sectors.

Eleanor Williams is a public policy and evaluation professional and was previously the Director of the Centre for Evaluation and Research Evidence at the Department of Health. Eleanor holds a Masters of Public Policy and Management and Masters of Evaluation from the University of Melbourne and has worked in a variety of senior management and executive roles in Government and as a management consultant with KPMG. Eleanor is the current convener of the Victorian Regional Network Committee of the Australian Evaluation Society and founded the Australian Public Sector Evaluation Network.

Launch of AES First Nations Cultural Safety Framework

Presented by Sharon Gollan and Kathleen Stacey with Sharon Clarke and members of the AES Indigenous Culture & Diversity Committee

The AES First Nations Cultural Safety Framework is intended to outline principles, provide practical guidance on roles and responsibilities of different stakeholders and what contributes to culturally safe evaluation and identify the outcomes that can be achieved by implementing the Cultural Safety Framework.

2021 AES Awards for Excellence in Evaluation

The AES Awards for Excellence in Evaluation aim to recognise and share excellence in evaluation theory, practice and use by:

• motivating people to come into and stay in the field of evaluation

• promoting the use of evaluation for positive impact and social good, for example supporting positive

• outcomes for Indigenous communities

• promoting advances in the field of evaluation.

The Awards recognise leading evaluators, leading evaluations, evaluation best practice and emerging evaluators.

The six AES Awards for Excellence in Evaluation are each focused on specific areas of the evaluation field:

• Evaluation Systems Award

• Indigenous Evaluation Award

• Evaluation Journal Of Australasia Publication Award

• Public Sector Evaluation Award

• Emerging New Talent Award

• Award For Enhancing The Social Good

We are excited to announce the Award recipients for 2021.

Dr Farida Fleming is the Chair of the AES Awards and Recognition Working Group. Farida is an evaluator with over 20 years of experience in supporting social justice initiatives locally and internationally. Farida provides monitoring, evaluation and learning technical advisory services at a range of levels – strategic, thematic and programmatic.

2021 Australian Evaluation Society AGM

Chaired by John Stoney, President 

Join the Australian Evaluation Society (AES) Board as we celebrate another year’s achievements by members of the AES, and introduce the new Board.

Followed by the announcement of new Fellows of the AES by Rick Cummings.

A whirlwind introduction to systems evaluation

Facilitators: Andrew Hawkins, Julie Elliott, Brian Keogh, Lewe Atkinson

The purpose of this presentation is to provide a very brief introduction to the world of systems evaluation and to invite members to webinars and other work of the Systems Evaluation Special Interest Group (SIG). We have kept it to four short presentations to avoid overwhelming people and to keep the energy high.

The four co-chairs of the SIG will each present for 5 minutes:

• Andrew: Systems Evaluation compared to other types of evaluation

• Julie: Waves and Maps for systems and complexity

• Brian: Systems Leadership

• Lewe: Results-Based Accountability - an entry point and one straightforward approach to systems thinking for evaluation in action.

We will then finish with a brief summary of the vision and mission of the SIG. The vision of the SIG is for evaluators to be able to pivot between prevailing evaluation practices and those informed by the language, concepts and theories of systems thinking, and the fast-growing sciences of social complexity.

Our mission is to facilitate a respectful discourse that provides space to discuss different conceptions of systems with a mindset of a mutually disputatious community of respectful truth seekers committed to finding common ground and exploring differences.

Finally, we will take any questions and invite people to join the community.

Andrew Hawkins: Andrew is a Partner at ARTD Consultants where he has worked for the last 14 years as a full-time evaluator. Andrew has a deep interest in the theory and practice of evaluation – and is interested in the evolution of evaluation from a research-based discipline to a broad-based way of thinking about the value of action in the world.

Julie Elliott: Collaborator, evaluation scholar-practitioner and performance analyst in complex operating environments. Taking the position that human interaction is always complex, Julie hopes to spur on evaluators to become clearer about the manifestation of systems or complexity sciences they are appealing to, how they understand it in social terms and transfer their ‘systems thinking’ and/or ‘complexity thinking’ framework of ideas into practical evaluative action.

Brian Keogh: For the last eighteen years, Brian has been a management consulting in water, social service organisations, the arts, and small to medium businesses. Prior to this he led some of the most innovative teams worldwide in the arts. He is partners in Cobalt59.

Lewê Atkinson: Global Partner of the Haines Centre for Strategic Management Limited in Brisbane since 2011. Lewe is described as a unique innovation professional with a solid technical and financial grounding which, combined with his systems thinking super-powers, enables him to see patterns, pathways, and solutions that others don’t. He has been selected as member of the Ian Potter Foundation Evaluation Pool (TIPFEP) on the basis of excellent financial analysis abilities with a focus on Social Return on Investment.

Evaluator competencies self-assessment Part 2: Using the results!

Facilitators: Amy Gullickson, Taimur Siddiqi, Lauren Wildschut, Sarah Mason, and the AES Pathways Self-Assessment Working Group

Last year at FestEVAL we alpha tested the AES Evaluator Competencies Self-Assessment - thanks to all of you that participated! We're back this year with two sessions. In this second session, we’ll look at the individual and organisation level self-assessment reports (if you haven’t done the self-assessment, we’ll have an example you can review). We’ll have time for you to review the individual report and give us your feedback - What do you want to see in a report about your competencies as an individual? We’ll also discuss how organisations could use self-assessments and hear from you about how AES could make the most of the self-assessment.

Amy Gullickson is the Director of the Centre for Program Evaluation at the University of Melbourne.

Taimur Siddiqi is an evaluation consultant and Managing Director of The Incus Group.

Lauren Wildschut directs the Evaluation Studies Programme at the Centre for Research on Evaluation, Science and Technology at Stellenbosch University

Sarah Mason is the Director of the Center for Research Evaluation at University of Mississippi

The AES Pathways Self-Assessment Working Group

Building the Aboriginal Evidence Base - self-determination and evaluation approaches

Facilitators: Gabrielle Johnson, Dana Pyne, Stephen Bennett

The Victorian Aboriginal Children and Young People’s Alliance (the Alliance) and the Victorian Aboriginal Child Care Agency (VACCA) play important roles in supporting culturally strong, safe and thriving communities in Victoria. As part of this endeavour, VACCA has have been leading the Building the Aboriginal Evidence Base project in partnership with the Victorian Department of Families, Fairness and Housing and the Alliance to further develop evidence of what practices and programs are culturally appropriate and achieve measurable outcomes as determined by Aboriginal communities.

This project is a commitment and critical enabler of Wungurilwil Gapgapduir, a tripartite agreement between the Aboriginal community, Victorian Government and community service organisations which is aimed at improving outcomes for Aboriginal children and young people. Wungurilwil Gapgapduir seeks to develop the capacity of the Aboriginal sector to build an evidence base and share knowledge.

This work supports self-determination objectives and recognises that both program design and evaluation must recognise context, community and place. This presentation will share insights from the initial stages of the project and reflect on how these early findings can be applied in evaluation approaches more broadly and provide examples of how VACCA is building an Aboriginal evidence base of their programs and practice.

Gabrielle Johnson is a proud Wemba Wemba and Wiradjuri woman, with 18 years’ professional experience working in Aboriginal organisations in the child and family welfare and family violence areas since 2001. Gabrielle has been a case worker and program manager and is now works within the Outcomes and Evaluation team of the Victorian Aboriginal Child Care Agency (VACCA). Gabrielle was the Senior Project Officer for the Aboriginal Health, Wellbeing & Safety Evaluation Framework which VACCA led in partnership with Melbourne University. Gabrielle is currently Acting Manager of the Cultural Therapeutic Ways Project, an organisation wide approach that guides VACCA’s work with families, Community, and staff.

Dana Pyne is the Executive Manager of the Victorian Aboriginal Children and Young People’s Alliance (the Alliance) and has worked with the Aboriginal communities in Victoria for over 9 years. The Alliance groups 15 Aboriginal Community Controlled Organisations (ACCOs) registered to deliver child and family services to their communities. In 2014, we formed the Alliance, with the belief that together we are stronger and with one unified voice we can positively influence the future for Aboriginal children and young people living in Victoria. The Alliance has a crucial role in driving change to achieve better outcomes for Aboriginal children and young people.

Stephen Bennett is a Principal Policy Officer in the Evidence Strategy Team, Community Services Quality and Safety Office, Victorian Department of Families, Fairness and Housing. The Evidence Strategy Team drives the translation of evidence-based policy and practice to improve outcomes for Victorians. This includes working in partnership with VACCA and the Alliance in implementing the Building the Aboriginal Evidence Base project to contribute to Objective 4 of Wungurilwil Gapgapduir to build and share Aboriginal knowledge, learning and evidence.

Our Place On Country - Aboriginal Outcomes Strategy, NSW Department of Planning, Industry and Environment (DPIE)

Facilitators: Robert Kelly and Chanelle Burman

This presentation will cover the journey taken by the DPIE Aboriginal Strategy & Outcomes Team in developing a MERI framework and Aboriginal Outcomes Integrity Approach for the Our Place On Country Strategy (OPOC).

Robert Kelly is a Yirandali and Ngati Maru man, who currently lives on Latji Latji Country and works on Paakantyi kiira kiira (Barkindji Country). He is the Manager for MER in the DPIE Aboriginal Strategy & Outcomes Team.

Chanelle Burman is a proud Thungutti woman, who currently lives and works on Dharawal Country. She’s also the Manager for OPOC Strategy in the DPIE Aboriginal Strategy & Outcomes Team.

FestEval Quiz night

Facilitated by Julie Elliott on behalf of the AES Canberra Regional Network Committee 

What is the collective term for a group of evaluators? How many evaluators does it take to change a light bulb? In the recent census, how many households registered an evaluator at their residence? Why do so few 5 years olds want to become an evaluator when they grow up?

Feeling fed up, bored and lonely cooped up in lockdown? Then this is for you! 

The Canberra Regional committee are hosting a hilarity-packed FestEval evaluation quiz and you are invited. All you need to do is take off your mask, get teamed up with your  evaluation buddies, brush up on your evaluation trivia and prepare for a fun and friendly competition.

We look forward to seeing you there and may the best team win. 

Julie Elliott: Collaborator, evaluation scholar-practitioner and performance analyst in complex operating environments

FestEVAL Club

The FestEVAL Club is a relaxed place to drop in and chat with colleagues. Our facilitators and MCs will be there to manage the proces. The Club is strictly BYO – grog and bean bag – so settle in for as long or little as you like!

Thursday 

“NOT only for the die-hards”
Back by popular demand, this will be an evening of unstructured conversation and good times … “Come out to the coast, we’ll get together, have a few laughs”

Re-imagining multicultural evaluation - taking our practice to the next level

Facilitators: Professor Rodney Hopson, Dr Katrina Bledsloe, Dr Jane Davidson and Ms Eva Sarr

This lively, interactive think tank invites evaluators interested in multicultural evaluation in our region of the world to reflect on the following questions: 

• What are the strengths of multicultural evaluation in the areas where you work? 

• What are the challenges of multicultural evaluation in the areas where you work?  

• What are our aspirations for multicultural evaluation? 

• What do you think you need to equip you to undertake evaluations that authentically respond to local cultural contexts?  

The primary aim is to generate an initial agenda for lifting multicultural evaluation in the Australasian region up to the next level. A secondary goal is to connect like-minded individuals who will be able to network and support each other in enhancing their own practice in multicultural evaluation. 

Rodney Hopson, Ph.D. serves as Professor of Evaluation in the Department of Educational Psychology, College of Education, University of Illinois-Urbana Champaign. Previously, he served as Professor, Division of Educational Psychology, Research Methods, and Education Policy in the College of Education and Human Development at George Mason University and as Hillman Distinguished Professor, Department of Educational Foundations and Leadership in the School of Education, and teaching faculty member in the Center for Interpretive and Qualitative Research and Honors College in the School of Liberal Arts at Duquesne University. He received his Ph.D. from the Curry School of Education, University of Virginia with major concentrations in educational evaluation, anthropology, and policy, and sociolinguistics. Rodney is a past President of the American Evaluation Association (AEA), and co-founded and led the Association’s Graduate Education Diversity Internship program for a number of years. He serves as core faculty of the University of Illinois, Urbana-Champaign’s Center for Culturally Responsive Evaluation and Assessment (CREA).

Katrina L. Bledsoe, Ph.D. is an internationally known leader in cultural contexts, cultural responsiveness in evaluation, and theory-driven evaluation in community-based settings. She has published extensively on the practice of cultural responsiveness and equity in the conduct of evaluation within communities ( In addition, she has led evaluation projects utilizing a multi-method approach for federal agencies such as USAID and the Substance Abuse and Mental Health Services Administration (SAMHSA), and has worked with US-based foundations such as The Robert Wood Johnson and Annie E. Casey Foundations, and community-organizations such as Johnson C. Smith University’s Smith Institute. Katrina is a mixed methodologist by training and in practice and specializes in quasi-experimental and mixed-method designs. Her expertise is in community-based education, health and mental health and social services program evaluation, applied social psychology and cultural responsiveness, and equity in cultural contexts. Dr. Bledsoe has conducted evaluations in African nations such as Kenya and Ghana as well as in North America. Dr. Bledsoe served a three-year term on the Board of Directors and was a member of the task force that developed AEA’s Public Statement on Cultural Competence in Evaluation. Dr. Bledsoe has spoken for US government agencies such as the Office of Management and Budget, and for international evaluation associations such as the The Aotearoa New Zealand Evaluation Association.

Jane Davidson: is an internationally recognized evaluation specialist, best known for developing evaluation rubrics as a methodology for getting the right “values” infused into an evaluation and ensuring they are applied in a systematic, transparent, valid, and credible way. In 2021, the values we apply should include the ones relevant to the two most important crises of our time - (1) equity and (2) the health of our climate/natural system. How do we do that, in a practical way? Jane will showcase how she has been doing this in recent work with organisations working at evaluation’s cutting edge, such as the Europe-based Laudes Foundation. Laudes now uses a coherent evaluation and learning system to drive and track systemic and structural change. The result was a creative blend of a systemwide theory of change (ToC), rubrics to track progress against the ToC, and mixed-method evidence to assess progress against the rubrics. Jane is Honorary Principal Fellow with the University of Melbourne and former Associate Director of the internationally recognized Evaluation Center at Western Michigan University, where she launched and directed the world’s first fully Interdisciplinary Ph.D. in Evaluation. She was the 2019 recipient of the American Evaluation Association’s prestigious Paul F. Lazarsfeld Evaluation Theory Award.

Eva Sarr: Eva is from a CaLD and multi-denominational background. She is a sixth generation Scottish, Irish, African Australian who was raised by a Muslim father and Catholic mother. Eva is the CEO and Founding Director of Centre for Multicultural Program Evaluation, based in Melbourne Australia. She is the founding chair of the Australian Evaluation Society’s first Multicultural Evaluation Special Interest Group and one of two affiliate evaluators with the Center for Culturally Responsive Evaluation and Assessment at the University of Illinois, Urbana-Champaign. Eva is a mixed methods evaluator who has worked as a monitoring and evaluation specialist for 12 years. First in international development, then in the NT, with the Victorian PS and now with a broad range of stakeholders who work in the multicultural and equity space.

AES Mentoring Program Pilot, early evaluation findings and next steps

In 2020 the AES Pathways Committee ran an innovative new pilot mentoring program for early to mid-career evaluators. This innovative program matched selected participants to one of four mentors who led monthly online mentor sessions for their small groups over six months. A formative evaluation of the program was conducted by a team of volunteer evaluators. Early findings about the mentoring program, its effectiveness, relevance, worth and replicability from the perspectives of mentors, mentees and the Pathways Committee will be shared along with thoughts for the program's future in 2022 and its wider application for professional networks in a COVID locked down world.

Julie Elliott: Julie describes herself as a collaborator, evaluation scholar-practitioner and performance analyst in complex operating environments. 

Jill Thomas: Jill is an experienced evaluator and analyst, who has worked in the health, higher education, and finance sectors in major cities and far northern Queensland. Jill has specialist skills in working with organisations to establish performance monitoring and evaluation frameworks, conduct program evaluations, design, data and process analysis and implement evaluation capability building programs.

Evaluators navigating ethics 

Presented by Lisette Kaleveld, Keryn Hassall, Keren Winterford, Racheal Laugery, Samantha Abbato, Eleanor Williams

This participatory session will allow evaluators to explore questions of ethics and practice with others interested in considering the challenges and issues that arise. This session will commence with a recap of the panel session held on Evaluators navigating ethics session held on Wednesday and some discussion starters to support interactive discussions in breakout rooms with space to report back and share findings.

The purpose of the two seminars is to provoke critical thinking on the topic of ethics and provide space for practitioners to share their experiences and reflect on their practice.

Lisette Kaleveld is a social researcher based at the Centre for Social Impact (UWA), and has also worked for the public service, the not-for-profit sector and as an independent consultant. She has conducted policy reviews and evaluations in mental health, disability, homelessness, the arts and education sectors. Recently, Lisette has worked with more focus on bringing in co-design methodologies and working with lived experience expertise as an important dimension of knowledge and evidence, especially when dealing with complex social problems. Lisette has also worked as a journalist, has a Graduate Diploma in Journalism, a Bachelor of Science (Anthropology), and is Convener of the Western Australia Regional Network Committee of the Australian Evaluation Society.

Keryn Hassall is an evaluator working across many policy areas. She uses ideas from philosophy, public administration research and organisation theory to inform evaluation practice. Her consultancy work is primarily formative and process evaluation helping government agencies to clarify program theory, understand the context, improve program design and implementation, and build the craft skills of public administration to support program effectiveness.

Dr Keren Winterford has more than 20 years work experience in the international development sector, in multiple capacities with Managing Contractors, NGOs, as a private consultant, and more recently in development research. Keren's areas of expertise include research design in partnership, particularly with NGOs; participatory research; facilitation and training; design, monitoring and evaluation; and strengths-based approaches to development. She has a passion and commitment to ethical research practice. She has sectoral experience and expertise related to gender and social inclusion, citizen participation, governance and social accountability.

Racheal Laugery is the Acting Research Ethics Manager at the University of Technology Sydney (UTS) and has worked in the area of research ethics for over a decade in a variety of roles, covering human ethics, animal ethics and biosafety. She provides individualised expert advice to academics and HDR students, not just through the preparation and submission of ethics applications, but throughout the research lifecycle.

Dr Samantha Abbato is an experienced evaluator and trainer for the Australian Evaluation Society (AES). Sam has more than twenty-five years of experience and strong methodological expertise across a range of qualitative and quantitative disciplines. Her academic grounding in quantitative methods is built upon a Bachelor's degree in mathematics and physiology and a Masters of Public Health in epidemiology and biostatistics. She has extensive qualitative training in medical anthropology (Ph.D., UC Berkeley). As an independent consultant, Sam has assisted a range of clients, including non-government organisations and local, state, and Commonwealth governments. As the Director of Visual Insights People since 2013, she has introduced a pictures and stories approach to evaluation for the health and community sectors.

Eleanor Williams is a public policy and evaluation professional and was previously the Director of the Centre for Evaluation and Research Evidence at the Department of Health. Eleanor holds a Masters of Public Policy and Management and Masters of Evaluation from the University of Melbourne and has worked in a variety of senior management and executive roles in Government and as a management consultant with KPMG. Eleanor is the current convener of the Victorian Regional Network Committee of the Australian Evaluation Society and founded the Australian Public Sector Evaluation Network.

FestEVAL Closing Session

In this session John Stoney will reflect on 2021’s week of FestEVAL as well as his time with the AES Board since 2016.

Followed by the launch of the Adelaide 2022 Conference theme by members of the Conference Committee.