Risks to your ERP-SAP implementation

1. Inadequate as is documentation Symptoms: You are the implementation Project Manager for a consulting firm and you have a client that just selected an ERP system. You (the project manager) and your team start gathering requirements from end users through focus groups, workshops, sessions with SMEs, etc. After gathering information from end users you erroneously conclude that you have all the necessary information and requirements to successfully implement the to be software system. Since you believe you understand the to be system you neglect to document the as is system. During UAT (Users Acceptance Testing) you find out that the system does not meet end users expectations and the UAT participants cannot validate your implemented solution. Your client is dissatisfied with the proposed solution, and you are now challenged to prove that your implemented ERP solution is consistent with the clients business needs, and requirements. Given clients complaints you are put in a position where you cannot explain how the proposed ERP solution meets or exceeds previous system functionality. The question now becomes how does the ERP system meet replaced system functionality if at all? Suggestions: Determine if there are any existing documents in the form of functional specs, architecture diagrams, requirements or flow process diagrams that describe current system functionality. Work with the client and in particular programmers who designed the legacy systems to understand the nuances and high-risk areas of the system that will be replaced. Document from your findings your understanding of the current as is system and also specify how you will replace the as is functionality with the to be ERP system. Draft test cases that specifically address how the replaced functionality will be verified within the ERP system and also allow the UAT members to peer-review the drafted test cases as they become available. Document any feedback from the UAT members for the drafted test cases and as time allows produce prototypes of the proposed solution to minimize the impact on the end users.

2. Requirements not scrubbed Symptoms: Organizations in particular within the federal government draft hundreds or thousands of requirements that are often phrased as the system shall. Instead of focusing on what the requirement should do the requirements should focus on what the system will do. Frequently government organizations implementing ERP solutions draft requirements ahead of making a decision to acquire an ERP system and thus the requirements are incongruent with the capabilities of an ERP system.

These government organizations implementing an ERP solution document requirements that are often maintained in web of spreadsheets that makes it difficult to: 1. Track a requirement, 2. Modify the requirement while communicating the changes to the affected parties, 3. assigning requirement ownership, 4. Create an RTM (requirements traceability matrix), and 5. Manage the lifecycle of the requirement. The ERP implementation partner is tasked with interpreting the requirements from spreadsheets and discerning how these requirements will be implemented during realization and verifying that the requirements have been met during testing. The ERP implementation partner for the sake of meeting deadlines rushes through the blueprint phase does not scrub the requirement and blindly attempts to implement the requirement even when the requirement is not feasible, necessary or consistent with the functionality of the ERP application. After testing is finalized and the ERP system is deployed much debate ensues as to the functionality of the ERP system because it fails to meet the requirements and the end users lose confidence on the deployed ERP application. Suggestions: For companies undergoing requirements based testing obtain a requirements management tool such as requisite pro or caliber-rm. Scrub all requirements, ensure thorough understanding of the requirement and evaluate the requirement based on these attributes: feasibility, necessity, consistency, completeness, priority, correctness, traceability, and unambiguousness. Illustrate understanding of the requirement with flow process diagrams and through peer review sessions. Requirements that cannot be implemented or not feasible should be waived or listed as exceptions. Requirements that fall out of scope need to be communicated to the design and functional team and evaluated for impact. It is unlikely that your requirements will ever reach a perfect state since that could trigger a paralysis for your project but requirements need to be clearly defined and understood to allow your team to proceed with an acceptable level of risk. Once an acceptable level of risk is established the functional team can proceed to configure the system and the technical team can develop custom programs. Requirements need to be drafted for all areas of the SAP implementation and not just surrounding functionality. Requirements categories include: security, workflow, reports, interfaces, forms, performance, archiving, usability (section 508), etc. Define all stakeholders for the requirements and assign ownership for the requirements for each area. For requirements that are automatically met from the out-of-the-box ERP solution and do not need test cases they need to be clearly demonstrated and presented to the client. Merely presenting how the ERP system meets the requirement out of the box is not sufficient and waivers should be

signed off to indicate that the client accepts these requirements as being met as is. Draft an RTM (Requirements Traceability Matrix) that ensures that all requirements get coverage and get mapped to a test case. No requirement should be left orphan. 3. Vendor software problems Symptoms: When either testing or maintaining your SAP software you will find errors, enhancements, or bugs within your software that cannot be addressed with your current project team. These errors, bugs, and needed enhancements do not originate from having customized or implemented SAP incorrectly but are rather triggered due to a deficiency with the vendor software. Depending on the severity of the problem the impact to your business could be manifested in different manners such as: a. Client/end user dissatisfaction, b. inability to roll out a specific planned system functionality, c. financial losses, and d. Unstable system The vendor software problems can go unresolved for prolonged periods. Furthermore, lack of controls, participation from the SAP client and audit trails can cause the software vendor problems to erroneously become closed when in fact they were never resolved. Suggestions: Assign a point of contact from your SAP implementation to monitor and follow up with all OSS notes, suggestions and patches from the vendor. The assigned point of contact should document all attempted resolutions to fix the problem and whether they were successful. The point of contact should also produce and distribute a daily report of all software vendor problems to avoid having problems fall off the edge of the earth due to no responses from the client side after the vendor suggested a solution to the problem. The SAP client should also assign a priority level and discuss expected resolution dates to the software problem with the vendor. For instance being unable to run Payroll in production could be catastrophic to the business and should have a successful resolution within 4 hours whereas not being able to generate a form could be a problem with a workaround that can wait 5 days for resolution. 4. No Scope Verification Symptoms: I have seen acrimonious and bitter relationships between the ERP implementation partner and the client over the implemented ERP solution. These contentious relationships between client and implementation partner stem from the fact that the client feels that the implemented ERP solution does not cater to their business needs based on the documented scope, and the end users cannot perform tasks that were easily executed within the legacy systems.

The problem is compounded when the client report defects, short comings and bugs against the ERP system that were not part of the scope or documented via a requirement. When the ERP integration partner labels the end users reported defects and problems as enhancements rather than problems with the implemented ERP solution the relationship between the implementation partner and the client takes a turn for the worse. Suggestions: Implement a system of accountability, sign offs and hand offs for deliverables and work products. A client sign off implies acceptance and satisfaction with the deliverable. Disagreements over deliverables and verified requirements need to be documented and resolved. I have been in situations where after the SAP implementation partner delivers functionality per the agreed upon software requirements, the go-no go decision is made with support from the client and the system is deployed into production only to have the client complain that the system does not meet its needs without specific and concrete evidence. Client statements such as the system does not do x, the system is not as good as our legacy system, the system is too complex, etc are vague and meaningless. The client needs to provide specific examples as to why the delivered solution does not conform to its business needs and requirements before providing sign-offs or permitting the system to go-live. The implementation partner and client need to agree on the exit criteria before the testing phases are finalized and on the results from the User Acceptance Test (UAT). It is highly recommended that the UAT participants be permitted to actually execute hands-on a subset of the previously executed integration test scenarios. When the UAT participants accept the results from the UAT phase they are in fact validating and verifying the final solution. The UAT is a suitable opportunity to bring up perceived concerns and problems with the solution before its moved into production. Many companies do not place enough emphasis on UAT and rather rely on system demos to prove that the solution in fact meets user requirements without allowing the actual end users to interact with the system.

5. Test Tools become Tumbleweed Symptoms: Many companies spend half a million dollars on test tools only to have the test tools gather dust. Some companies that I have consulted for have indicated that they do not want to automate business processes because it takes too long to automate processes or they believe that the test tools are incompatible with their systems, or they had bad experiences with automation in previous releases, etc. Most of the time if not always these aforementioned excuses are devoid of merits. Spending half a million on an automated test tool solution that gathers dust does

not maximize return on investment and places undue pressure on the functional team to test the system manually. Suggestions: Establish an automation framework and hire qualified individuals to build a library of recorded test scripts that can facilitate regression testing, and future software releases. No matter how much test tool vendors dumb down their tools you will still need skilled testers who understand programming, development, data driven parameters, correlation, embedding error handling logic, and visual basic concepts since many test tools standardize on visual basic. If your project is attempting automation for the first time set up realistic automation goals for instance 40% automation of all business processes before the first release is due for deployment. The objective should be to build a library of recorded test scripts that can be recycled for multiple releases and thus maximize the ROI on your acquired test tools. Once you build a library of test scripts you will find that in fact test tools can shorten the test execution phase, while providing more consistent, reliable and greater coverage for your requirements over manual testing. Other uses of test tools include supporting your regression testing needs during standard production maintenance support and for loading training data. The alternative to test automation is manual testing which is susceptible to these problems: Coordination of resources along with their conflicting schedules Expenses associated with bringing resources to the site where the manual testing is occurring Lengthening the period of time need to conduct integration testing Manual testing is not repeatable thus if test scripts need to be re-executed you need to repeat all testing activities with resources who are suddenly unavailable You will not get automatically created test logs that most test tools produce and now you will have to rely on manual test logs which your testing team produces with mixed results. The objective is to recognize and realize that test tools can become your friend to ensuring reliability and dependability in a production environment while decreasing reliance on manual testing efforts. Companies that are most likely to be affected by the absence of automated tools are those who are highly regulated in FDA environments and those companies that have SAP variant configuration needs since they have to test their ability to build products with multiple combinations and permutations (i.e. when ordering a car over a website there are many combinations for configuring a new car).

6. Ambiguous Capacity Testing requirements and goals

Symptoms: I have been to many projects where performance, volume, stress testing are an after through and only marginally considered during the final prep phase. Companies neglect to draft consistent, necessary, complete, and thorough testable requirements for conducting capacity testing during the requirements gathering phase (blueprint). Other issues surrounding capacity testing that I have seen at other projects include unclear goals and objectives such as the system should run as fast as possible, the system should handle as many concurrent end users as possible, conduct a capacity system in a non production box and extrapolate results, etc. Suggestions: Determine within the blueprint phase of your SAP implementation the type of traffic, and business volumes that you are likely to see in your production environment. Leverage off statistics from your existing legacy systems, expected number of named users in production, and service level agreements (SLAs). Draft testable requirements that support your business needs for instance merely stating I need to have average response times of 3 seconds per screen to complete a sales order is unclear in contrast the statement under a 500 user load in addition to batch jobs, all screens related to sales order creation will not exceed an average response time of 4 seconds per screen 95% of the time when accessed via corporate LAN under the web enabled interface is a much more robust requirement. The project should document SAP user community profiles, along with expected system throughput from functional users, jobs running in the background and foreground, reports, batch jobs, RFCs, etc. The idea is to create distribution diagrams that show the systems peak usage under a typical day, peak hours of the day, end of the month activities, end of the quarter activities, and seasonality. Armed with these diagrams the performance engineer can develop the necessary automated scenarios to identify your applications bottlenecks and degradation points, while verifying all of your capacity requirements. No matter how thorough your capacity requirements are there will always be unaccounted traffic that is not emulated since capacity testing is not exact science. To make up for this short coming verify that your system will successfully meet SLAs under a seasonality load, and then proceed to run tests with a load that is 125% greater than the maximum expected production load. 7. Knowledge walks out the doors Symptoms: You have contractors and consultants who have demonstrated knowledge of your business processes and the ERP software. These contractors spend countless hours configuring the system, developing interfaces, reports, testing the system, and solving trouble tickets for end users.

Due to projects deadlines and priorities your contractors follow a loose, disorganized and unstructured approach for documenting their work deliverables and work products. Once your contractors receive better offers they walk out of your project. A few months later internal and external audits come to your projects and you are unaware of what your contractors accomplished, how they accomplished their work, and how their results were stored if at all. Your project loses credibility and faces a crisis in attempting to reestablish what the departed contractors accomplished during their tenure. Suggestions: Although high consulting turnover is invariably inevitable at most ERP implementations there are some pointers that you can observe to mitigate the damage that the departing resources cause to the project (assuming voluntary withdrawal from the project). Establish a repository (i.e. shared drive) where stored documents and artifacts follow strict naming standards. Create a QA (Quality Assurance) team whereby the QA representatives enforce project standards and guidelines. Mandate weekly status reports from all contractors where all accomplishments are reported. Set up meetings at least 2 to 3 days before the contractor departs to review work products and areas of high risk. Obtain contractors contact information before he departs in the event of project emergencies. 8. Disconnected transport management tool Symptoms: You are in production maintenance support mode and you have to regularly or at scheduled intervals transport objects from the testing environment into the production environment. Your company needs to generate reports, track and monitor the necessary approvals to transport an object. For audit and historical purposes you need to have the capability to generate on the fly reports to identify how an object was transported, how it was tested before it was transported, who approve the transport of the object, when the object was transported, etc but you cannot do so because your transport management tool was either built in house and is not integrated with SAP or you purchased an SAP implementation tool from an implementation partner that does not integrate directly with SAP, and is missing version control capabilities. Your internal processes and methods provide limited visibility into how objects were transported, is bogged down with ineffective approvals, and fails to facilitate your independent audits assessments. Suggestions: Obtain a tool that integrates directly with SAP and enhance or complement SAPs internal transport management system. The tool that integrates with SAP should only allow the Basis administrator to transport an object when all necessary approvals have been granted and enforce your projects business rules for transporting objects. In contrast, custom made tools or other commercial tools that do not integrate with Mercury allow the Basis administrator to transport

an object even when the necessary approvals are missing. Furthermore the selected tool should allow the end user to generate ad-hoc and custom reports and provide version controls capabilities which most in-house created do not offer. A tool such as Mercurys Change Management Extension for SAP Solutions helps you streamline the transport process and provide your project with Visibility, security, and auditability across multi-server SAP system landscapes 9. Undocumented assumptions, risks, constraints Symptoms: I have been to many ERP projects where at the beginning of the project a lackadaisical and perfunctory attempt is made to document project risks, assumptions and constraints. These risks, constraints and assumptions are stored somewhere and fall of the edge of the earth. When project woes surface (which they will) no mention is made to documented risks, assumptions and constraints which would aide in answering how managerial decisions were made and derived. Chaos ensues when the ERP implementation partner cannot stand on its own two feet and defend its decisions made in conjunction with established risks, assumptions, and constraints. Suggestions: Merely documenting risks, constraints, and assumptions is a start but not the end. Risks need to be constantly evaluated, assessed and revisited. Because a risk has not occurred in 2 months it does not mean it cannot resurface with a greater intensity and impact to the project in month 3. Risk management shows us that a risk can have secondary and residual effects. Adopt a policy of reviewing the risks with highest priorities at least once a week. Set a budget and necessary resources to confront the risk. Communicate with individuals affected most by risk. Review all project assumptions and constraints and document how these helped you reach your managerial decisions.

10. Neglecting the end user Symptoms: One of the most overstated attribute of an ERP system is the promise of real-time integrated data. However one of the biggest nightmares that end users find with an ERP system is integrated data. When closely examined we see that the promise of integrated data can deprive end users from making data adjustments or on the fly changes that were common with legacy applications. Other common problems that ERP end users face is finding data, producing the right reports, increased complexity, mis-configured workflow, ill-defined processes for performing critical tasks, recording of data in different places, etc.

ERP systems can drastically revamp the work of end users and adversely affect a companys mission critical processes when the end user input is ignored. Symptoms: Involve the end user early on during the requirements gathering phases and confirm with end users that requirements have been correctly captured and understood. Elaborate understanding of requirement with flow process diagrams. Obtain end user sign off and approval on captured requirements before proceeding to design a solution. For complex business functionality to be replaced with the ERP solution hold prototype sessions to demonstrate the systems capability, ease of use, and conformance to business rules and requirements. Solicit feedback from end users during the prototype sessions and ensure that all end users concerns and questions are addressed in order to minimize impact of the proposed solution in production. Additionally provide pre-selected User Acceptance Testing (UAT) participants with integrated test scenarios which allow peer reviews. The objective is to include the UAT participants early during the design phase and not just turn over a system that will be blindly tested during UAT. 11. Deficient Impact analysis and regression testing Symptoms: Production end users call the help desk to report errors and problems with their system functionality. The SAP production maintenance support group has the responsibility for addressing and resolving the reported help desk problems. The projects change control board and integration manager do not consider and assess the merits of the production problem, how many man-hours will be needed to address the problem, whether the production problem is scopecreep or consistent with documented requirements, and the total dollar cost to implement a resolution to the problem. Furthermore the problem is exacerbated when the SAP production team with limited or no documented analysis implements the software fix and moves it into production without considering how the transported fix will impact previously working system functionality. No end to end regression testing affecting other modules is conducted to ensure that nothing is broken after a fix is implemented even when the fix requires configuration changes. Suggestions: Establish a change control board (CCB) to analyze the impact of all system changes triggered as a result of problem reported to the production help desk. The stakeholders from the CCB need to review all proposed changes no matter how trivial or how minor for scope, conflict with existing requirements, priority, level of effort, testing needs, release notes, training notes, availability of resources, cost to the project, impact to the project schedule and necessity. As an example of impact analysis, after reviewing a reported production problem the CCB may decide that it is not cost-effective to spend 200 billable hours at an

average rate of $180/hr to resolve a production problem that requires configuration changes, user exits, etc when the problem is expected to be resolved in a future SAP version that will be released within the next 6 months. If the CCB on the other hand decides to implement a solution to a reported problem then the need arises to identify a list of sunny day scenarios that should always execute successfully after a configuration change is made. Automate the identified sunny day scenarios and execute these scenarios whenever a production change affecting the system configuration is scheduled to be released into production. 12. Hidden Scope Statement Symptoms: The Project Scope a vital and critical component to the success of a project which documents the projects deliverables and output is outdated, stored in multiple locations, or stored somewhere in many SAP implementations. It is natural that given this scenario many SAP projects experience scope creep, or create much confusion over what SAP functionality needs to be delivered for the customer. I have witnessed many SAP projects that engaged in either gold plating or fail to deliver needed functionality because the functional and technical team did not have access to the project scope or the project manager failed to maintain the project scope in a single location with version control. An inaccessible, hidden, or outdated project scope has the potential to increase project costs, decrease project quality, increase delivery times, and fail to meet customer expectations. Suggestions: Place the scope statement in a single repository with a version control tool that is readily accessible to all project team members. The rule of thumb is that anything not documented within a scope statement is not considered part of the project scope. The PMBOK recommends that as the project progresses, the scope statement may need to be revised or refined to reflect changes to the scope of the project. The scope statement should be reviewed and approved by the project sponsor and the customer

Scope statement is a documented description of the projects output or deliverables. According to the PMBOK from PMI:

The scope statement forms the basis for an agreement between the project team and the project customer by identifying both the project objectives and the major project deliverables. The scope statement as described by Jeb Riordan,

forms the basis for agreement between customer and supplier will be the basis for all project related decisions Will be used to determine whether the project has been completed.

In relation to the knowledge areas in the PMI PMBOK , developing the scope statement is included in and the most important output from the scope planning process. According to PMI a scope statement should include, either directly or by reference to other documents: project justification, project product, project deliverables, and project objectives. 13. Diluted Sign Offs - Waivers/Exceptions for sign offs Symptoms: Various groups and teams provide sign offs to formalize the end of a project phase or to demonstrate completion of a milestone and deliverable. The projects agreed upon sign offs for deliverables are provided by members of the project team and independent parties. The sign off is supposed to indicate that the stakeholders and owners of the deliverable approve the deliverable and that it has been successfully completed according to a predefined criteria and checklist. The value of the sign off is diluted and undermined when stakeholders and owners approve the deliverable with exceptions and waivers. These exceptions and waivers and their corresponding risks are never documented and they eventually linger on without a resolution. As an example I have been to several SAP projects where the system has gone live even after the User Acceptance Test participants do not agree with test results, and defects with priority 1 and 2 are still outstanding. These projects have made SAP go/no go decisions with exceptions and issues that are never resolved. The signs off to go-live are provided even though the system does not meet the necessary go/no go criteria and instead project managers provide sign offs for the go/no decision to make an artificial project deadline. These actions in essence negate the value of the sign off process. Suggestions: Clearly define the criteria for providing sign offs for deliverables. Also provide the ownership for a deliverable and the intended end user of the deliverable. For instance the development team can program interactive reports for end users but this does not mean that the development team owns the deliverable rather they have been assigned to implement the deliverable.

For software releases, integration test efforts, cut-over activities the integration test manager and test manager can construct exit and release criteria that must be met and certified with independent parties in order for sign-offs to be granted. Any sign-offs that must be granted with exceptions and waivers must be documented with rationale for the exceptions in addition to timetables for resolving the exceptions and the owners for the exceptions. Furthermore an analysis should be conducted and documented for the consequences and risks that the approved exception and waives will present to the project. Sign-offs should come from independent parties and also from the owners of the deliverables with few if any exceptions to avoid undermining the authority and value of the sign-off. 14. Procedures, Standards, Guidelines are not adhered to or enforced Symptoms: The Quality Assurance team, the change management team, the integration team and cut-over team set up procedures and standards for: a. Creating test cases, b. Creating test scenarios, c. Creating BPPs (Business Process Procedures), d. Creating functional design specs, e. Creating technical design specs, f. Transporting objects, g. Change control, h. Reporting defects, i. Logging end user help desk tickets, j. Ground rules for holding workshops during blueprint, etc Yet these procedures and standards are not enforced. Common symptoms from failing to enforce project standards include: i. The functional team being unaware as to what the naming standards are for functional design spec documents, ii. the Basis team is unsure as to what all the necessary approvals are for transporting an object, iii. the test team does not know what the entrance and exit criteria is for moving from one testing phase to the next, iv. the change management team does not have clearly defined guidelines for creating BPPs, v. the functional team does not accurately capture end user requirements during blueprint, vi. User Acceptance Test (UAT) participants do not know their roles, deliverables and responsibilities during for testing the system.

vii. Development team does not create custom program that conform to project standards for performance, and naming conventions Suggestions: Document all project standards and guidelines in a shared drive after obtaining buy-in and approval for these standards and guidelines from the project stakeholders and PMO. Dedicate a resource to coach and train project team members on the projects standards and guidelines in particular for new resources joining the project. This resource should inspect work-products and deliverables at a predefined frequency. In one of my previous SAP implementations the project adhered to the Capability Maturity Model (CMM) which many SAP consultants were unfamiliar with and the project had dedicated resources to coach new arriving resources on the CMM model for SAP testing. Artifacts, deliverables and work products that are no in compliance with the projects standards and guidelines should be brought to the attention of the individuals responsible for producing the deliverable for corrective action. 15. Unclear contents for templates---No Standards Symptoms: The SAP ASAP methodology is replete with templates and implementation partners will offer a plethora of templates for testing, drafting test cases, gathering requirements, documenting specifications, etc. Providing or handing down a template from one team to another does not address a critical issue what level of detail is necessary to fill out the contents of a template? Given this issue its natural to see how even the best templates fail to capture the necessary information for requirements, specifications, and test cases. As an example one of my previous projects had templates for drafting detailed requirements, and specifications which were deliverables during the blueprint phase and the various functional teams drafted the detailed requirements and specifications to their best of their abilities. When independent reviews of these deliverables were conducted the auditors determined that many of the functional teams had deficient and insufficient contents and information for their specifications and detailed requirements. The independent reviewers determined that the templates did not meet minimum criteria, and the deliverables needed to be re-written. Much time was wasted in rewriting the specifications and requirements which delayed the blueprint phase and increased project costs.

Suggestions: Templates are terrific for helping the project team collect and document information. However a better approach to just handing out templates would be to provide standards, guidance and instructions for filling out the contents of the templates. Set up a kick-off meeting where all affected parties get specific instructions for filling out the contents of the templates. Provide samples of well constructed templates to the audience. Also provide the criteria that will evaluate the quality of the filled out templates. Do not wait until the due date to review all the templates. Templates should be evaluated early and gradually as they become available which allows for corrective action without much impact to the project schedule in the event that team are not complaint with the established standards for filling out the templates.
Conclusions: The author exposes and reveals perils to the SAP implementation

that need to be addressed as early as the requirements management phase. The suggestions for overcoming these perils include involving SMEs for capturing requirements, scrubbing requirements, end user hands on execution during UAT testing, transport management tool, and advocating the use of automated test tools. The author based on his experiences has witnessed how one or more of the perils above has crippled other ERP projects and believes that planning for these risks with proven techniques will increase the chances of success at most ERP implementations.