MEL March 2022

The Government of PNG and the Government of Australia, through the PNG partnership, are committed to strengthening the Monitoring Evaluation and Learning (MEL) capacity of the PNG Law and Justice Sector at all levels from executive to middle managers and officers. Hence, A multi-year Monitoring Evaluation and Learning Professional Development Activities Plan has been developed to enhance Monitoring and Evaluation in the Law & Justice Sector by the Law & Justice Sector Secretariat (LJSS) and Justice Services & Stability for Development (JSS4D) Program.

The Law and Justice Sector Secretariat in partnership with support from the Australian Government through Justice Services and Stability for Development (JSS4D) is committed to supporting change by building a critical mass of personnel with MEL knowledge and skills at all levels across the agencies. 

LJSS managed the first in a series of workshops called ‘Introduction to Monitoring Evaluation and Learning’ at the Lamana Hotel in Port Moresby from 08^TH March 2022 to the 10^TH March 2022. It was an interactive workshop facilitated online by Square Circle.  It covers all aspects of MEL as it relates to participant’s work. The workshop among other key M&E elements of training merges theory with practice through a relevant Melanesian case study.

Square Circle is a global development consulting company headquartered in Brisbane, Australia. Working at the intersection of theory and practice, Square Circle offers services in collaborative investment design and adaptive programming; leadership and policy; governance; institutional strengthening; digital storytelling; research; monitoring, evaluation and learning; and conflict, peace and development.

Facilitators

Participants

Department of Justice & Attorney General (DJAG)	Jairus Yetigo – CP, Fidelma Kara – PPM, Francis Irere – PPM, Benjamin Maragau – CP, Evelyn Jarua – CP, Helen Bua – CP, John Vegala -PPM, Sheila Kumanga – PPM, Wanu Gau – GESI, Goa Tau – LJSS, Racheal Lo – LJSS, Maryanne Natung – LJSS, Miriam Lakame – LJSS, Dale Urame – LJSS, Tommy Wama – LJSS, Marie Roa – LJSS
Constitutional Law Reform Commission (CLRC)	Angela Ranama, Bugia Ghani
Independent Commission Against Corruption (ICAC)	Murphy Nick, Tamara Laveil Sapulai, Annie Samuel
Legal Training institute (LTI)	Douglas Beeu
PNG Correctional Service (CS)	Joseph Sila, Talitha Appa
PNG Magisterial Service (MS)	Melvina Wagi, Tinut Kali, Francis Timan
Office of the Public Prosecutor (OPP)	Abraham Motowaya, John Pandi
Office of the Public Solicitor (OPS)	Natalie Tamate
Ombudsman Commission (OC)	Natasha Piniau, Thaddeus Tiriman, Ninipe Gurumo, Teio Same
Prime Minister & National Executive Council (PMNEC)	Ware Aulakua
Royal Papua New Guinea Constabulary (RPNGC)	Steven Dama, Apishane Tore, Gabriella Sipelung

The attendants were good mixture of Law & Justice Agencies with more than one person representing his/her Agency where over the course of the three (3) days, the following content were covered through an intensive three days program:

DAY 1 – 08TH MARCH 2022
Time	Session	Facilitator/Presenter
8:30 – 9:00	Registration, Coffee and Tea
9:00 – 9:20	Opening and Program Overview	Facilitation team
9:20 – 10:05	Getting to know you; Getting to know the topic	Facilitation team
10:05 – 10:50	MEL Baseline knowledge and Needs	Facilitation team
10:50 – 11:00	Morning Tea
11:00 – 11:45	Theory of Change: An Introduction	Facilitation team
11:45 – 12:30	Theory of Change in Practice	Facilitation team
12:30 – 13:30	Lunch
13:30 – 15:00	Theory of Change Case Study and Role Play	Facilitation team
15:00 – 15:15	Afternoon tea
15:15 – 16:00	Theory of Change Presentations	Facilitation team
Day 1 Close

DAY 2 – 09TH MARCH 2022
Time	Session	Facilitator/Presenter
8:30 – 9:00	Registration, Coffee and Tea
9:00 – 10:00	Log frames and Results Based Management: An Introduction	Facilitation team
10:00 – 10:45	Turning a Theory of Change into a Log frame Logic Part 1	Facilitation team
10:45 – 11:00	Morning Tea
11:00 – 12:00	Turning a Theory of Change into Log frame Logic Part 2	Facilitation team
12:00 – 13:00	Lunch
13:00 – 14:30	What is Program Logic	Facilitation team
14:30 – 14:45	Afternoon Tea
14:45 – 16:15	Applying Program Logic to our Case Study Theory of Change	Facilitation team
Day 2 Close

DAY 3 – 10TH MARCH 2022
Time	Session	Facilitator/Presenter
8:30 – 9:00	Registration, Coffee and Tea
9:00 – 10:30	Data Collection Methods: Interview Techniques	Facilitation team
10:30 – 10:40	Morning Tea
10:40 – 12:00	MEL Methods and Tools Part 1: Designing Interviews MEL Methods and Tools Part 2: Designing Surveys	Facilitation team
12:00 – 13:00	Lunch
13:00 – 13:45	MEL Methods and Tools Part 3: Inductive Approaches	Facilitation team
13:45 – 14:30	MEL Methods and Tools Part 4: Analyzing Existing Data	Facilitation team
14:30 – 14:40	Afternoon Tea
14:40 – 16:10	Monitoring vs Evaluating	Facilitation team
16:10 – 16:45	Evaluations & Workshop Close

Methods Square Circle used to facilitate the workshop/training:

• Online throughout the duration of the three days
• Each attendee was given an account to access the MEL online learning platform to post activities done during the workshop, information exchanged and presentations of the presenters where also accessed.
• PowerPoint slides and short videos were posted for participants to review/download
• Provided each attendee with a workbook (Monitoring, Evaluation and Learning Workshop 2022 Workbook)
• The MEL online platform was still active and updated after the workshop with more learning materials until March 31^ST 2022

By the end of the 3^RD day of the MEL workshop, a survey was conducted, and the results collected are from the attendees who took the survey were:

Few other questions were surveyed to find out whether the training:

• Had achieved its objectives
• Had an impact after the three days of training/workshop in terms of knowledge and skills building, learning and or improvements by the participants.
• If the training methodology used was good and successful
• Whether the training materials and resources provided, content and intentions were fully utilized, appreciated, understood and utilized by the participants

Parts of training/workshop participants found useful:

Categories	Score	%
Developing Logical framework, Program Logic and Theory of Change	22	62.9
Collecting & analyzing data using technology	3	8.6
Developing Logical framework, Program Logic and Theory of Change together with Data Collection & Data analysis using technology	2	5.7
Learning by doing activities and working in groups	3	8.6
Refresher to forgotten M&E processes	1	2.9
The entire training methods & activities	4	11.4
Total	35	100.0

The majority, 62.5% of the 35 attendees who took the survey found the Development of Logical Framework, Program Logic and Theory of Change to be very useful to them in relation to the work they do. Overall, the majority of participants either agree or strongly agree that what they have learned during the workshop will be useful to them in the work they do either:

• Enhance their performance
• Broadening their understanding on Monitoring & Evaluation
• Increase their outcomes/outputs and deliverables in Monitoring and Evaluation
• Broadening their perspective on Monitoring & Evaluation including activity planning with elements of Monitoring & Evaluation

Various feedbacks were received from the participants and were categorized as follows:

57.1% of participants indicated that the MEL Workshop/training should involve more interactive exercises and group discussions where Monitoring & Evaluation can be related to PNG context. More explaining on some of the concepts were also needed and facilitators should have been physically present. The remaining 47.9% suggested that:

• The training needed more time and the time should have been managed properly during the 3 days
• More course materials required and improvements on the online platforms should have been made and more technology should have been integrated into the training/workshop

What changes will the participant do as a result off the training?

Answers received from participants were also grouped in categories as shown in the table below:

Categories	Score	%
Improved processes involved in data collection, analysis and reporting	6	17.14
Developing M&E systems	9	25.71
Upscaling M&E capacity, systems and functions	6	17.14
Participant did not answer the question or the answer provided is not related/irrelevant to the question	1	2.86
Utilizing M&E tools and technology to improve current M&E processes and more improved and broader M&E practices and applications	13	37.14
TOTAL	35	100

62.85% of Participants strongly indicated that one of the outcomes of the training/workshop enhanced their skills and understanding more broadly on how to utilize M&E tools and technology to improve the current systems, practices and processes they were using at their Agencies, including development of M&E systems. This implies that the M&E processes and systems the participants were using was not so effective and changes were needed, such as upgrading, upscaling and improving the older systems and processes.

34.28% of the participants indicated that they identified the need to upscale M&E capacity in terms of technology, systems and functions as well as improving on data collection and reporting, where they relate technology in the ICT space of significance.

As a result of the training/workshop, participants were able to identify weaknesses in their older M&E systems, practices and processes and can rectify or implement changes to theses themselves.

Further Comments:

Comments	Score	%
Agencies Updating each other on their respective M&E roles after the training/workshop and include more people to participate in M&E	2	5.71
More training/workshops and include more agencies	13	37.14
Workshop/training in the further to be more physical than virtual	4	11.43
More case studies and examples in the next workshop/training	1	2.86
Continuity of Community of Practice	4	11.43
The training was very useful and advanced my M&E skills	1	2.86
No further comments, have better understanding of M&E and appreciate the training	6	17.14
Facilitators should improve on delivering the contents	3	8.57
Did not answer the question	1	2.86
TOTAL	35	100.00

From the survey conducted, the following can be concluded:

• More than 75% of participants found the training to be very useful and helped them understand Monitoring & Evaluation and its practicality
• More than 75% of the participants agree that the Monitoring, Evaluation and Learning training/workshop enabled them to acquire new innovative thinking and approach to Monitoring & Evaluation
• Realizing the importance of planning, monitoring, data collection, analysis, evaluation and reporting
• Better and improved understanding of the different techniques and processes of Monitoring & Evaluation
• Incorporating technology and M&E tools for improved monitoring and evaluation processes
• Implementing learned content into practical use

Participants also appreciated and enjoyed the training very much and strongly recommend that more M&E training be conducted for the Law and Justice Sector Agencies and the continuity of the community of practice be maintained and strengthen.

Overall, it was a much-needed workshop and the outcome was achieved where participants themselves now have better understanding of Monitoring & Evaluation, its processes, systems and its practicality; they are now able to relate them, put learned content into practical use and achieve their set targets and outcomes by setting objectives, planning activities and implementing the Monitoring & Evaluation Cycle to it.

The training/workshop was concluded with participants receiving their certificates from Square Circle.