Evaluation of the ProjectTracking System at RMG

Helen Miller Evaluation

Table of Contents
ABSTRACT ............................................................................................................................. 3 INTRODUCTION ..................................................................................................................... 4 PROGRAM DESCRIPTION ....................................................................................................... 4 EVALUATION METHOD .......................................................................................................... 5
Participants ................................................................................................................................................................................ 5 Procedures ................................................................................................................................................................................. 5

Results .................................................................................................................................. 6
Online survey results ....................................................................................................................................................................6 Interview Results ...........................................................................................................................................................................8 Quantitative Results .....................................................................................................................................................................9

Discussion ............................................................................................................................. 9 EVALUATION COST .............................................................................................................. 12 APPENDIX A ........................................................................................................................ 13 APPENDIX B......................................................................................................................... 13 APPENDIX C ......................................................................................................................... 17

ABSTRACT
Real Estate Management Group (RMG) contracted Helen Miller Evaluation to conduct a formative evaluation of the new project-tracking system put into place by the Creative Services Coordinator. This system was set up to organize the overwhelmed communications department at RMG eliminate a three month backlog of projects and prevent such a backlog from reoccurring. The system was also set up to improve interdepartmental communications and to create an archival system for digital design projects. The evaluation included surveying the employees who most frequently need services of the communications department, interviews of key senior staff members, interviews of the communications department staff as well as quantitative results from the projects done in the last three months. The evaluation found that the project-tracking system is, in fact, helping to organize the communications department, eliminate the backlog, and is improving interdepartmental communications as RMG hoped it would. Results from surveys and interviews showed that although there has been improvement in workflow and communications there is a resistance to the change in policies by many staff members. The author of this report recommends more communication to staff members of the importance of the new policies and strict adherence to it if they are to continue to keep the communications department organized.

INTRODUCTION
Real Estate Management Group (RMG) is a privately held organization which manages real estate and property management associations. A major portion of the business revolves around the management of Internet and graphics communications for these associations. In recent years RMG has found difficulty in keeping the communications department organized. There is no formal system for requesting or tracking communications projects. Due to a high designer turnover rate and a lack of organization the communications department RMG is dealing with a three-month backlog of projects. This backlog is causing systematic problems throughout RMG as many employees rely on the materials created by the communications department to do their jobs. To deal with this problem RMG has hired a Creative Services Coordinator (CSC) to reorganize the communications department. The coordinator has been tasked to: 1. Set up a system to assign and track projects. 2. Help to reduce the backlog and make sure to avoid such a backlog in the future. 3. Create a system to digitally archive graphics projects so they are easily retrievable. 4. Aid in interdepartmental communications. RMG hopes to expand its communications services to offer more a-la-carte graphic communications via print, web and social media to their clients and to the general public. In order for RMG to expand their services they need to get the communications department organized and running efficiently. RMG has requested Helen Miller to evaluate of the systems set in place by the Creative Services Coordinator to ascertain if the systems are working to organize the communications department and to see if they should continue to invest in expanding the services offered by this department. This document is the final evaluation report of the system set up to organize the communications department at RMG. This report contains a description of the system the Creative Services Coordinator set up, a description of the evaluation method and procedures, the results of the evaluation and the final recommendation for RMG.

PROGRAM DESCRIPTION
The CSC created a system to address the problems in the communications department. Although the system is meant to be a permanent solution, the implementation of the system took place over the course of three months. The system was created to solve the issues outlined by the upper management of RMG as well as issues observed by the CSC. The goals of the system are to improve internal communications, organize and track communications projects, increase department efficiency and cross-train designers to enable them to work on all clients. The system set up by the Creative Services Coordinator has four major parts to it. 1. The project request form (see appendix A). An HTML form was set up on the company intranet for staff members to fill out when they need any services from the communications department. The form was created to aid in internal communication

between departments as it tells the communications department what type of project is needed (print, web or e-blast), the specifics about that project (quantities, budget, timelines) and is delivered to a generic email address which is then forwarded to Creative Services Coordinator who then assigns the projects to designers based on task as opposed to client. If the CSC is not in, the email can be forwarded to someone else in the communications department to ensure the project is processed. The processing consists of assigning each project a number and a designer. 2. Projecttracking sheet. Utilizing Google Docs a project-tracking sheet was created to track projects, their due dates, dates completed, project requestors, and other necessary information. The collaborative nature of Google Docs allows for the designers in the department to prioritize projects based on due dates and to communicate any issues or problems with each problem. In addition to having a central location with which to track projects, it provides a means to improve the internal communications between the designers. 3. Digital storage. All communications projects were moved into one centralized location and named based on their client and project number. The goal of this was to make it easy for designers to find and store projects and in effect increase department efficiency. 4. Assigning by task, not project. To cross-train designers on the intricacies of different clients and to play to the different strengths of the communications staff projects were assigned by task as opposed to client.

EVALUATION METHOD
Because the focus of this formative evaluation is to ascertain if the systems put in place by the CSC are effective in the goals set out by RMG to organize and increase the efficiency of the communications department Helen Miller evaluation chose to evaluate using a goal-based model. The evaluation used mostly qualitative methods to measure the organization and efficiency of the communications department of the new policies set in place by the CSC. Data were collected through 1. Interviews with staff. 2. Attitude surveys of staff. As a quantitative measure Helen Miller Evaluation used the projecttracking sheet to measure the amount of projects and deadlines met and or missed since the tracking system was put in place.

Participants
RMG has a staff of about 40 people many of which have little or no contact with the communications department. Participants for interviews and surveys were chosen by their frequency of use of the communications department. Helen Miller Evaluation identified 10 people who used the communications services on a regular basis to survey and interview including the members of the communications staff.

Procedures
Staff members were sent an online survey (survey can be viewed in Appendix B or online) to assess their attitudes about the new project request form and policies of the communications department. The survey consists of 10 items asking respondents to rate their attitudes towards the new policies initiated by the CSC and the how those policies effected their work and communication with the communications department.

Helen Miller Evaluation identified five key staff members who mainly used the services of the communications department to gain insight into how the new system affected the work-flow through their departments. The members of the communications staff were also interviewed to see how the new system affected their ability to work. The project-tracking sheet (Appendix C) was used as a quantitative measure to see how many projects the communications staff were assigned and if they were able to keep up with due dates. This will give RMG concrete data as to the amount of and efficiency of the program.

Results
Online survey results When asked on a scale of 1-5 how happy were you with the ability of the communications staff to deliver projects on time before the new policies set forth by the CSC took place, 80% responded they had been very unhappy, and 20% were unhappy, noting they had to wait unreasonable amounts of time to get their projects.

When asked on a scale of 1-5 how happy they were with the ability of the communications staff to deliver on projects on time after the new project request system 80% responded they were very happy and 20% were happy and noted they were getting their projects in a timely manner.

When asked on a scale of 1-5 how confusing do you find the project request form to be?

70% of respondents did not think the form was confusing and 30% thought it was somewhat confusing. Some of the comments mentioned that the

form itself was not confusing but that confusion came in about when they should fill out the form.

When asked on a scale of 1-5 how difficult was it to communicate your project needs to the communications department before the project request form? 80% of those surveyed said it was either difficult or very difficult to communicate with the communications department and 10% or one person thought it was not difficult at all.

When asked how difficult it is to communicate with the communications department now 90% responded that it was very easy and 10% said it was somewhat difficult. When asked what they found difficult it was noted that they were unsure of whom to discuss their project with after submitting a project request form. It was also noted that they sometimes felt the communications staff was short with them.

When asked do you prefer the new system to the old one? Why or why not? 60% preferred the new system to the old one noting they are pleased with the turn around time of their projects. 30% were not pleased and noted they preferred simply emailing a request to a designer.

The survey asked if they thought the project request form was missing anything. More than half of respondents thought the questions on the form were fine but wanted an email confirmation and copy of the form they filled out.

When asked if they felt the new system had been explained to them adequately there were mixed responses. Comments suggested they were confused about why such a system was needed.

When asked how they felt about the new system overall comments ranged. Some staff noted they were happy they were getting their projects in on time but confused why the change was necessary. Several people noted they felt the communications staff was working as more of a team now, and some expressed frustration to change in general and were not happy with the new system. Interview Results Helen Miller Evaluation first interviewed the two senior members of the communications staff about how they felt about the new project request and tracking system as well as the new file organization system. The communications staff was exceedingly enthusiastic about the new system. They felt they had better control of their files and spent less time looking through the company server for files. They liked the way requests came in with necessary information already included because it cut down on the amount of time they had to spend trying to track that information down. They also thought it helped them to prioritize their own workflow and liked that the form explained to staff that they needed more than an hours notice to get projects done. There were frustrations expressed by the number of people who tried to skirt the system and email requests as they had done in the past. They also noted that they still felt frustration about the amount of time they were given to complete projects. They mentioned that despite explanation the staff only seemed to give them one day or less to complete projects and on one day they received seventeen requests all with a one-day turnaround time. They also expressed that aside from the basic information in the project request form it was often difficult to get content from the staff to complete their work. The evaluator also interviewed five key staff members who utilize the communication department services extensively. Overall they were pleased with the turn-around time of projects and the elimination of the backlog of projects which was holding up a lot of their work. They thought the interdepartmental communications had improved slightly. A senior staff member said that she would not use the project request form during convention season because she would be too busy to do so. They mentioned there were still many staff members uncomfortable with the change but they felt that the change was necessary. These staff members also wanted access to the projecttracking sheet so they could see what was happening in the communications department. The management also seemed confused about what designers were working on which clients. 8

Quantitative Results The project tracking and request system began on February 25, 2011 and was tested for three months. In that time out of 183 projects 79% of those projects were completed on time or early. Of the projects which were not completed on time, most were waiting on the requestor to approve the proof, supply content, held up due to equipment issues or because of illness of a designer. In this time the three month backlog of projects was also completely eliminated.

Discussion
The purpose of this evaluation was to see if the program initiated by the CSC was successful in accomplishing the goals set forth by the management at RMG. The goals were to organize the communications department, increase interdepartmental communications and create a system to track and archive graphics projects. Based on the survey results the project-tracking and requesting system appears to be working. Before the system was set in place 100% of those asked were at least unhappy with the turn around time of projects in the communications department. When asked how happy there were after the system was set into place 100% were at least happy with the turn around time of projects. This shows that they system has been helpful in aiding the communications department to prioritize and get work done on time. When asked about the ease of use of the project request form 90% felt at least comfortable with it while 10% felt somewhat confused by it. Based on comments the confusion was more about when they should use the form and not the form itself. This suggests that the staff is somewhat confused by the process and not the tool itself. RMG form after submission. Interdepartmental communication was addressed in the survey. 90% of respondents felt that before the system was put in place it was very difficult to communicate with the communications staff. They noted that in the past they were often told no when requesting a project or got attitude from the communications staff. Since the implementation of the new system 90% now feel that it is very easy to communicate with the designers in the communications department. 10% still have difficulty communicating with them. These results suggest the project request form appears to be helping with interdepartmental communications. The survey also questioned participants about if they preferred the new system to the old one. Although the previous results suggest the system is working only 60% did prefer the new system. Comments suggested the staff did not understand why the new system was necessary and that they were uncomfortable with the change in policy. They preferred emailing a request to the designer. The evaluator interviewed several key staff members. The interviews with the designers in the communications department showed they were very happy with the new system set in place. They felt they were better able to prioritize their own work and felt less frustrated in general. They expressed frustration with staff members who seemed

unwilling to follow the new process and the amount of time they were given to complete projects. They also were happy with the new project archiving system. They felt they spent a lot less time searching the server for projects and more time getting work done. Overall they felt their department was organized and able to provide better service. Interviews with the staff outside the communications department were also conducted. These staff members were satisfied with the improved turn around time of projects and the elimination of the backlog but wanted access to the project-tracking sheet. Although she conceded the system was improving the workflow throughout the office a senior member of the management staff mentioned she did not think she would use or insist on the use of the project request form during the busy season. This suggests that she does not feel it is necessary. The biggest issue noted throughout these interviews was that many staff members did not understand or feel comfortable with the change in policies. The quantitative date showed that 79% of projects since the implementation of the new system were completed on time or early. It also showed a total elimination of the backlog of projects. This shows that the new system is working to improve workflow and that the communications department is able to keep up with projects. The project tracking system seems to be working based on the goals set out by RMG although there is room for improvement. This formative evaluation has shown that the project request form is aiding the communications staff in prioritizing their workflow and delivering projects on time. Helen Miller Evaluation recommends the CSC set the form up to deliver a copy to the requestor for their records after submission. The project-tracking sheet also appears to be successful in tracking the workflow of the communications department. It has also aided the designers in communicating with one another about what is happening with each project and it is providing quantitative data about the amount of work the department is doing. The evaluators recommend the communications department set up a calendar to work with the tracking sheet for the rest of the RMG staff to be able to see the workflow happening in that department. This may help in staff understanding why the designers need more realistic turnaround times and aid in the transparency the staff desires. The archiving system for the communications department has helped in the efficiency of the communications staff. They mentioned they spend much less time searching for their projects and more time completing projects. Interdepartmental communication, although improved, is still an issue at RMG. Based on the survey and interviews the staff seems confused about why the changes are necessary and are very uncomfortable with the change in policies. Helen Miller Evaluation recommends RMG hold one or meetings to address these issues and explain to the staff why this change is necessary and important. Until this is issue is resolved there will continue to be discomfort amongst staff members. Helen Miller Evaluation also recommends that the senior management embrace and follow the new

10

procedures fully and insist the staff do the same. If not, it is likely the backlog of projects will return.

11

EVALUATION COST

12

APPENDIX A

13

APPENDIX B

14

15

16

APPENDIX C

17