Introduction

This Monitoring, Evaluation, and Learning (MEL) Foundations Course (Training) is a part of the Justice Services and Stability for Development (JSS4D) MEL Capacity Development Program in collaboration with the Law & Justice Sector Secretariat (LJSS) to enhance skills and embed MEL in the Law & Justice Sector Agencies. This includes identifying and collecting data, data analysis, developing and the implementing MEL framework, best practices, and communicating data as an integral part for officers in the Law and Justice Sector.

The overall goal of the Program is to develop the foundations of a sustainable culture of MEL in the Law and Justice Sector (LJS) – fundamental to enhancing Officers of the Law and Justice Sector with the skillset to intensify LJS Agencies’ performance, deliverables, and accountability to the law and justice service delivery in Papua New Guinea. The MEL Foundations Course (Training) is a suite of learning activities supporting the Program goal.

This MEL foundation was delivered by Square Circle Founding Director Dr. Kamil Shah (Education and Capacity Development), JSS4D advisors Ms. Helen Cherry and Mr. Neil from Australia all in persons, including POM-based PNG MEL Expert Ms. Joy Waffi at the Crown Residence. Square Circle is a global development consulting company headquartered in Brisbane, Australia. Working at the intersection of theory and practice, Square Circle offers services in collaborative investment design and adaptive programming; leadership and policy; governance; institutional strengthening; digital storytelling; research; monitoring, evaluation, and learning; and conflict, peace, and development.

Training and Learning Activities

The training and learning activities aimed to equip Officers of LJS Agencies to better understand the purpose and benefit of MEL in workplaces. To learn why, how, for whom, and with whom, MEL is implemented, develop the Theory of Change (ToC), and understand relationships between ToC and Monitoring Indicators along with Impacts Evaluations by enhancing Data literacy using relevant data in respective workplaces.

The training span over 5 days, from the 13^TH to the 17^TH February 2023 alongside the following activities:

Opening of the Workshop/Training

The training was officially opened by Deputy Secretary National, Provincial Coordination & Crime Prevention Division of the Department of Justice & Attorney General Mr. Stanley Raka.

Deputy Secretary Raka emphasized the value and importance of the training and elaborated further on the outcomes of the training consequently the repercussions concerning performance and change of approach towards achieving and delivering on outcomes affiliating to performance monitoring frameworks (PMFs), indicators, outputs, outcomes, data, and reporting. Deputy Secretary Raka expounded on the training and implored the participants to appreciate the substance of the training as it will revamp performance, and the daily data collection procedures and systems for reporting against PMFs through Coordination, Collaboration, and Communication of data and reporting employed through innovative platforms.

Deputy Secretary Raka acknowledged the continuous robust support of JSS4D to the Law & Justice Sector (LJS), including the partnership of development and establishment of the SLOS/LJS Database and M&E tools as the innovative instrument for M&E in the sector. He also commended the advisors and training facilitators for being available in person to deliver the training and not online closing off his remarks by accentuating advancement in performance, data collection strategies and methods, data analysis, and reporting of the participants after the training.

Training/Workshop

The training structure throughout its duration incorporated self-assessment activities and learning with an emphasis on recurring interactive activity sessions between participants and trainers.

Activities throughout the course	Outcome after completion of the course
Personal Learning Journey Journal: Written and/or video reflection Theory of Change Depiction and Presentation: Group task MEL Framework: Design the basics of a framework Data collection tool: A mock data collection and/or analysis tool to answer evaluative questions.	Understand the purpose of MEL applied to your specific work context Identify and engage with stakeholders of MEL Develop a Theory of Change (ToC), Evaluative questions, and design a MEL Framework Understand and implement a range of data collection techniques.

Self-assessment was daily basis by each participant to reflect on key takeaways and lessons learned throughout each day. Key reflection questions to associate each daily takeaway to their organization, role, performances, and practices concerning MEL:

Reflection on the key takeaways and lessons learned over the course at the end of each day

Why is MEL important for my organization? Why is MEL Important to my role? What skills do I have that can contribute to effective MEL? What skills do I need to develop further to contribute to MEL? How can I apply what I learn in the course to my work? How can I promote a culture of MEL in my agency? How can I be part of a MEL Community of Practice?

Learning and activities expounded on:

Participation and Feedback

From the initial confirmed list of 20 participants, only 18 attended. The daily attendance, participation and engagement of the 18 participants was overwhelming. They consisted of 2 from Magisterial Services (MS), 12 from various branches of the Department of Justice and Attorney General, 1 from JSS4D, and 3 from the Royal Papua New Guinea Constabulary (RPNGC), all from Port Moresby, who were very motivated, eager, energized and anxious to learn and maintained this positive vibe right from the beginning to the end of the training.

On the last day of the training, a survey was conducted with 17 participants who were present on their feedback about the overall training:

Did the trainer present the information clearly?

29% agree and 71% strongly agree that the trainer presented the information clearly, hence the information, in terms of content and context that was presented throughout the training was very clear.

Did you understand what the trainer/s said?

Around 6% of the participants were unsure/neutral/maybe whether they understood what the trainer/s said or not but 41% strongly agreed with 53% agreeing that they did understand what the trainer/s said. There might be various factors why the 6% did not understand what the trainer/s said such as hearing difficulty, not fully concentrating, loss of awareness, fatigue, being seated further away from the trainer, etc. Overall, the presentation was excellent because 94% of participants agree to have understood what the trainer/s said.

Was the rate at which the information was presented suitable for you?

94% strongly agree and agree that the rate at which the information was presented was suitable, whereas about 6% are unsure/neutral/maybe whether the rate at which the information presented was suitable for them. Therefore, it is concluded by 94% of the participants agree that the rate at which the information was presented was suitable. Factors affecting the 6% might vary from not following through, and fatigue to periodic concentration disturbances/disconnection, etc.

Did the trainer/s satisfactorily manage the group?

The trainer/s satisfactorily managed the group because 59% strongly agree and 41% agreed that the trainer/s satisfactorily managed the group.

Did you gained knowledge as a result of attending this training?

All participants concurred they gained knowledge as a result of attending this training, where 35% agreed and 65% strongly agreed to have gained knowledge as a result of attending this training.

Did you find the sessions useful and relevant to your work?

All participants found the sessions useful and relevant to their work where 71% strongly agreed including 29% agreed to the usefulness and relevance of the session in terms of relevance to their work.

Overall, were you satisfied with the quality of the training?

The overall quality of the training was satisfactory according to all participants where were 65% strongly agreeing to include 35% agreeing to satisfaction with the quality of the overall training.

Which part of the training did you find useful to you?	Why?
ToC	Because it’s related to what I do at my workplace.
	Because it’s relevant to my work.
	(No justification)
	I have learned how to identify the data that need to be collected and how they should be analyzed.
	There is a need for change in the way some of my work is done and with the knowledge gained I can use it to make changes.
Data Collection	I’m involved in data collection at my workplace.
Evaluation Methods	when activities are developed, I usually don’t ask these questions. This training has helped me to improve in that area.
Evaluation Methods	Evaluate Questions Differentiate formative and summative.
ToC and MEL Log Framework	Did not have a clue what they were. Heard from officers who do MEL work until I attended this training.
Case Study Activity	(No justification)
Focus Group Discussions	Because we learn from others too.
MEL Log Framework	I will be managing the resources effectively and identifying issues and resources that can be used widely.
Coding and analyzing	(No justification)
MEL Log Framework and Program Logic	because that is what I do at work, planning my annual work plan for the year and breaking it down into smaller activities that can be achieved daily, monthly, quarterly, half-yearly, and annually.
ToC, MEL Log Framework, Data Collection, Data Analysis, and Coding	(No justification)
Entire MEL Training Content	(No justification)
ToC and discussions based around ToC	(No justification)


*Which part of the training/workshop could be improved?*	Focus group discussion topic. Some more theory notes are required to demonstrate its relevance to data collection.
	Program logic. More examples need to be included to help participants understand how it is formulated and framed.
	Everything was okay, presented well, and assisted logically. Should elaborate more or in detailing with the Framework.
	Logical Framework Need more explanation on the different terms used in order to differentiate and apply them correctly.
	Training should materials be given to participants prior to before commencement of training
	Coding, by putting up the framework for it.
	Non.
	In my personal view if there can be a case study as an example for us to get a clear picture of how the Justice system can be monitored and evaluated.
	Data.
	Nothing that I can think it was very informative.
	Visual display and logistics of participants.
	Group discussions.
	Next training could you expound on how to use digitalization to collect and analyze data.
	If the course were distributed earlier, I would have read and prepared myself ahead.
	If we could be given a Wi-Fi password to connect to the MEL community.
	I can’t see any since all parts of the training were useful and worthy.
	Theory of Change. Because it requires a detailed explanation to really und the concept.


*What changes will you do in your work as a result of this training?*	Working closely with data.
	With my basic understanding, I believe I can now relate more to what happens in MEL. I get to appreciate the work of MEL. Also, in terms of my work, I get to now understand MEL and relate to the reporting templates.
	Utilizing what I learn into improving how we do Monitoring and Evaluation at RPNGC. The use of digitalization to collect data is an example.
	Training programs monitoring and evaluation leading to performance improvements of staff of Probation service.
	The skills and knowledge gained will help improve my job and deliver service to village courts and land mediators.
	Share what I learned with colleagues.
	Proper planning and implementation of activities whilst monitoring their success and then doing g analysis of it
	Practice more monitoring and evaluation.
	Mostly, I will be applying the skills I just learned in the areas of projects, programs and the policies since I directly involved in them.
	Monitoring and evaluation methods.
	Managing and utilizing my existing data well. Representing the data in a proper way that other stakeholders
	I will get more involved in the analysis, monitoring, and evaluation in order to work plan.
	Decision making. Contribute to Discussion. Speed up the M&E of Activity Plans. Evaluating the Reports, and the trends. Assists in implementing projects plans.
	Coding.
	Changes on the programs that I participate in, to have it improve and make work much easier.
	Basically, the template and information interpreted will be efficient meaning use resources wisely towards the area that is needed and misappropriation of funds.
	Apply the skills/knowledge in my own workspace in terms of appreciating program logic.


*Comments*	I wish to acknowledge JSS4D and Square Circle for the MEL training. I have learned a lot from this training.
	I am so delighted to be given this opportunity to be part of the group to attend this very insightful MEL training I learned so much. If any training in the future I’d love to attend.
	Non.
	A big thank you to the facilitators & JSS4D for funding this important training.
	All in all, it was a successful learning week. It was worth learning the mechanism of what really MEL Is.
	I would recommend that the duration of the MEL training extends or be part of DJAG’s annual corporate plan.
	In future workshops, more emphasis must be given to topics like the Theory of Change.
	I recommend continuing these courses.
	My recommendation from this training is that hardcopy of course materials should be made available in future training as some participants find it difficult to access using their devices.
	We need more training like this.
	Materials to be supplied in advance prior to commencement of training 2. Course materials should contain examples or templates, videos and other relevant materials to the topic taught.
	No further comments.
	I recommend that all DJAG officers should Attend MEL training.
	Non.
	Everything is clear and vital to the work that I’m doing. I’m grateful I am a participant in this training.
	No other additional comments.
	I really need more training on MEL.

It is coherent that the training was an all-inclusive success, overall, well delivered with the appreciation and benefit of success of this training having trainers on the ground delivering the training in person, where content and context were transposed smoothly and logically to the participants. Participants’ awareness, consciousness, and attention were all fully focused on the trainers, whereas while delivering training online, participants have difficulty understanding and following through attentively per the last MEL that was delivered online in 2022.

The entire training was conducted very effective, rational, objective and delivered very well in a chronologically logical fashion that was well digested by participants. Certain contents and activities of the training were particularly useful to individual participants as they provided insight and detailed elucidation into areas of engagements and processes that entailed enlightenment. The approach at which this training was shepherded by the trainers in delivering the content to the participants was marvelous as it combined learning through activities, peer-to-peer engagements, and direct trainer-participant engagement. The trainers were also experts in their areas of training; hence the content and context of the training were transferred to the participants smoothly in simplified terms and demonstrations including answers to questions being answered by each trainer per the relevance of the question to each trainer’s field of expertise.

This training has reinforced the participants’ knowledge of M&E and provided them with the proficiency to better understand and appreciate the MEL processes, how to adjust and enhance their performance to properly align their activities in such a manner that is objective, achievable, outcome-oriented, and properly executed by implementing MEL skillsets as tools and guides.

Conclusion

The training/workshop concluded on Friday the 17^THwith presentation of the MEL certificates by Deputy Secretary Raka. Verbal requests were made from participants during the closing ceremony for more MEL training, and inclusion of more participants, especially co-employees of the participants and other Agencies. MEL as they have come to know during the five days of training was an exposure to them in various aiding perspectives in the matter of individual performance advancement, deliverables, reflections and assessments. All participants were grateful and appreciated the capacity development initiative through the MEL Training by JSS4D and LJSS and highly esteemed the value of knowledge gained throughout the training.