Oracle ERP R12

Just another WordPress.com weblog

« R12 Upgrade – Top Ten Pitfalls
R12 – The Worst Oracle Release Ever? »

R12 Upgrade Plan

By irroberts

1. The most important part prior to starting an R12 Upgrade is to build support for the
project. If the Business isn’t on board, it’s going to hit real problems when you want their
time – and it’ll take a lot of their time……Look for the key drivers of why your business
needs an upgrade; for us we had many new Oracle modules and we felt it was better to do
those projects on R12. rather than implement in 11.5.10 and upgrade in R12 later. A
move to Linux and the age of 11.5.10 were other critical factors. With R12 being out for
2.5 years by the time we implement we figured it would be relatively stable overall – a lot
of companies got badly burned by jumping way too early into R12.

For those companies not in R12 now is a good time to start thinking – the product is
stable in most areas (even AP is getting there) and the market is depressed, so there are a
lot of available resources (albeit mainly in 11.5.10, but most modules remain pretty
similar overall)

2. A study approach of building a Business Case to assess the Upgrade may be a good
idea. It won’t cost a lot, but will allow you to make a very informed choice ; the new
functionality, the customizations you have, the estimate of effort, time and cash it going
to take, pitfalls and other issues.

This approach lets you build concensus and support. It minimizes risk and allows you to
build a strong business case to get necessary funding.

For us, we identified several major issues upfront; Payables was a complete re-
implementation for us. Payments was a complete re-implementation. We had a LOT of
customizations. General Ledger customizations require re-work; small but 60% of our
customizations were hit…..Self Service screens must be migrated. All Payment formats
require new build. This was going to be the biggest upgrade I have seen in a very long
time…..Understand also the architecture – Oracle ERP isn’t an island. It usually talks to
an awful lot of other systems and if you don’t understand this at the outset it may hit you
just before go-live or worse……after.

We also used a lot of good documentation on R12. Look on Metalink, google, OAUG.
People have made plenty of mistakes and published where they went wrong – with the
documents available now you should avoid the pitfalls for most.
Remember this is not just an upgrade – Oracle has brand new modules so this is also a
major implementation project; Payments is new and completely different, EBiz Tax is
new, Subledger Accounting is new and if you are running AP , you have no choice but to
implement the Payments. The other two are now core modules that must be implemented,
even if your not using Tax………

Be careful of your plans for current Production during the Upgrade. Most of your
resources will sink into R12, so try and reduce big new commitments.

3. Starting the project. Make sure you have enough hardware……you’ll need to run a lot
of instances AND still support your existing Production on 11.5.10…….If you don’t have
the hardware your project will suffer badly. Resourcing wise there are lots of options, but
it will take a lot more than an upgrade from 11.5.5 to 11.5.10. I can’t stress enough how
important it is to get top tier consultants – a few really good people (albeit expensive)
actually saves you money. Depending on your customizations, you’ll need a fair chunk of
development resources. But by doing the study you’ll be able to make good estimates.
(Don’t underestimate Payables and Payments – these will take a VERY LONG TIME).

Remember that resourcing won’t just be Oracle people; you will need people from the
feeder systems involved (Legacy Teams, Data Warehouse Teams – the stuff you change
may create work for them), the DBA Team, the Unix Team, the Email Team, the ERP
System Admins, potentially people from external (such as Citibank). Most of all , your
going to need your users.

4. So your ready to start. Once your over the ubiquitous kickoff meetings (which
shouldn’t have any surprises as you did all the planning, lobbying, studies in advance
right ? ), its time to get the team going.

Obviously an R12 ERP would be helpful. Make sure that you have the hardware, know
how to upgrade a copy of Production and have it available BEFORE you have your
Upgrade army. It’s a lot of cash to burn through otherwise….If you’ve got a good DBA
Team they should be able to get you an instance in 2-4 Week timeframe.

If you have good internal resources, then this really helps. Otherwise external consultant
will need to take a bit of time to understand how you use Oracle ERP – sure most
companies work in roughly the same way, but every company has its difference; some
are very different indeed.

At this point I normally split off into streams; I split into modules, team leads across each
module (we have 31 standard and custom modules, spanning HRMS, Procurement and
Finance).

The Functional Team should be preparing an initial high level Test Plan, working closely
with Business. In parallel they should use their knowledge (and the R12 Upgrade Guides)
to identify the relevant changes to the company. The Test Plan should be comprehensive
and signed off once complete. This should cover all Test Scenarios across all Business
Areas, including external systems. The Functional Team should also try to identify, in
conjunction with the Business and Development Team, customizatons that can be
removed with new functionality provided as standard. (We removed 10 check formats in
a single morning – imagine the saving when you need to rewrite these, do Technical
MD.070, Functional MD.050, Testing, etc. A Little time doing this could bring
significant savings).

The Development Team should again be split across Modules. If you have a
customization register (and we have all 1,600 customizations registered so we don’t miss
any) you can use this as a very good monitoring tool. Working closely with Functional
Team, identify the core stuff – there is no point in working on a report first, if you can’t
enter the invoice to test the report. Once you prioritize the customizations (and possibly
modules if you have limited resources), start working to upgrade these, document and
test.

Now based on R12 experience, from a Technical Perspective, AP and Payments are very
difficult. GL customizations had 60% small updates required. Self Service needed a re-
implementation to move from MOD*PLSQL to standard Java pages. Custom modules
were largely unaffacted and OAF upgrades with minimal overall issues.

If the Developers update the Customization Register with flags saying Unit Tested,
Fixed, Require Fix, it becomes an excellent monitoring tool providing exact progress on
your project. Ensure that you use source control. There are cheap, cheerful and excellent
solutions including Subversion. This is a great tool.

Once the Test Plan is signed off, Consultants and QA’s should start ensuring that
comprehensive Test Scripts are available. As they are being written, the application
should be tested initially to throw up the bugs. The earlier you find them the earlier you
get them to Oracle Support to get fixes. R12 does have quite a chunk of bugs, especially
in Payables. Keeping a close eye on Metalink is important – when new RUP’s are
released, you have a very important decision to make on whether to apply. If your well
into your project, the advice I would give is not to apply….they’ll cause a lot of issues,
it’s newly released and you’ll be the poor guys finding all the new bugs. Leave that to
someone else. Obviously if you have critical errors, pile the pressure on to Oracle to
provide single patches. If you are early in the project stage, then apply the RUP’s, but do
it on a Patch System and check it first…..they can cause real damage. At some point
though your going to have to freeze patches, otherwise you will never finish.

Whilst all this is going on your DBA team should be working on practise upgrades and
optimization. It will take many upgrades to get it right.

5. So you have your Test Plan, Test Scripts and Customizations. We run what we call an
Internal UAT. Basically its a UAT but it is carried out by the R12 Upgrade Team, not the
user. Try and get an exact copy of Production, upgrade fully, deploy the objects and do
the setup. (Be careful to carry out pre and post upgrade steps)
Given that some modules are going to be delivered before others, you may want to do
this staged. It’s not ideal but it does compress the overall project time. You will have
problems on Payables, and should that hold up your HRMS track? I would argue no.

The point of the Internal UAT is to run through the entire test, as the users would do. This
includes external systems such as Legacy, your planning systems, Citibank, or whatever.
All bugs should be recorded. (We use a tool called JIRA which is fantastic)

Be particularly careful to look at the upgrade process itself. You should simulate a close
on your R11.5.10, reconcile on 11.5.10, then do the upgrade then reconcile again on R12
and make sure the R11.5.10 and R12 add up. This is critical and we found what we think
is a major issue on accruals when we did this.

6. Another clone of Production should be taken. All objects, customizations, setup should
be done on the UAT environment (If you have not been keeping deployment registers,
BR.100 would strongly suggest you start doing so……) Usual UAT kickoff meetings
should be held and contact lists given to users. Your team will need to work hand in hand
with the Business Users to co-ordinate the UAT. It’s going to be a lot of work, a lot of
new and unfamiliar screens and quite a few complaints, no matter how well you’ve done
till now. The Test Plans and Test Scripts should have been agreed months ago, with full
review and signoff.

One trick here is to put the Test Plan on a shared drive and ask the user to mark
Pass/Fail/To Test as they progress. As an Upgrade Manager you can then review and
summarize easily progress across the entire ERP Suite. Now that is sweet…….Again
delivering some modules into UAT before others may be an option if you want to
progress quicker. Others prefer to deliver the entire UAT. What I have found is that by
progressing some modules quicker, it gives the project momentum and when you put out
weekly progress tables to users, they don’t want to be last to finish.

Make sure all Test Scripts are tested. Make sure Month End and Year End are similuted,
with proper reconciliations. Make sure all the systems to and from Oracle ERP are tested
and continue to work with R12. Make sure integration between modules is smooth and
working – organizing this across departments is tricky. A UAT room for this can be
handy…..providing cookies and drinks (non alcoholic please…..don’t want people testing
when they are drunk….) creates a good ambience.

Record all issues and bugs as you go – again a tool like JIRA is great. Monitor closely
and ensure that when your users are doing UAT, your R12 Team are there to assist. Some
areas may need twice weekly meetings on certain modules. This will help to manage
difficult UAT areas, stop user frustration and make the UAT a little less painful for all.
We did this on Payables in particular.

On completion, ask the head of each area and other users to sign off a standard AIM
Acceptance certificate.
7. Whilst the UAT was going on I hope you were also planning to do something else
right? As UAT goes on, you should be planning what will be a very compex and difficult
cutover. The team should also be doing rehearsals by taking copies of production,
upgrading, deploying, setting up and testing. All bugs should be recorded.

Don’t underestimate the deployment.

You should have deployment registers for each module, including tasks for DBA, System
Admin, etc. You should also have a big thick document on the steps the DBA’s do in the
actual upgrade. Separate deployment registers with hyperlinks to relevant setup docs,
BR.100, etc.

Review the deployment registers and figure out what can be done in advance of R12. We
transferred new programs, menus, value sets, etc to R11.5.10 without switching on
certain components. All of this saves a little bit of time and will ease the stress of the
upgrade weekend.

Ensure the BR.100 (or other setup instructions) are clear, concise and complete.

Call meetings with the users and make sure that the cutover (and possibly closing
processes) are clear. We are doing the closing in R11.5.10 just before cutover, closing
down all transactions, etc. Why? Couple of reasons. If data is left open in R11 (say
unaccounted) it will give real problems in R12. (We worked for months to get the script
from Oracle and luckily we found this prior to go-live). Secondly it allows you a whole
month before your first closing in R12. You can do a couple of simulations of Production
and sort out issues if needed. The users will lose Production for a few business days,
unless you do it on a holiday……..its a big upgrade. Our’s runs for 42 hours.

Get a register of who needs what responsibility on the day and how many setups they
need to do. Set these up in advance so you know who needs what access and are not
scrambling on the day. It also lets you balance the workload. We have almost 600 setups.
A lot in Payments.

Deployment of code. If you have used Subversion (or another Source control system)
then you can grab all changes and give to the DBA’s. Make sure your directories match
Oracle’s and Subversion as that will make deployment easier. A lot can be automated and
this will save a lot of time on the cutover. Now with all the prep work the DBA’s started
over the previous months maybe this is something they could also have been working
on…..There are a lot of standard Loaders that can save lots of time.

Talk to Oracle and make them aware of your key dates, cutover, month end, payroll, etc.
They can get you critical support management over that time.

Try and do as many rehearsals as you can whilst users are doing UAT. The closer the
rehearsal is to an exact recent copy of Production the better.
Make sure you do patch reconciliations between your UAT and Rehearsal databases.

Hold a few internal meetings to ensure that everyone in IT knows exactly their role.
Watch out for people booking holidays in advance. One key person out could derail your
go-live.

As PM, you should be writing a PM.030 Transition and Continegency Plan that covers all
the angles.

If the cutover fails all your hardwork will have gone to waste. Rehearse, Rehearse,
Rehearse……

8. Cutover

Generally over a four day window. Holidays are ideal but make sure you tell Business
well in advance if their key system is going to be down during business hours. We plan to
be down by 5:00PM on a Wednesday and up by 7:00AM Monday. We also have a
migration to Linux thrown in. Wednesday, Thursday are days for the actual Upgrade,
Friday the Upgrade Team roll in to do setup, customizations, etc. Saturday and Sunday
are for testing with a go-live decision on Sunday evening. As we are moving server, our
contingency is simple; if the cutover is a no-go we simply turn our old server back
on……Don’t repeat the errors you made in rehearsals, keep a log of everything that
failed and make sure it doesn’t happen on your go-live, as you simply won’t have time to
troubleshoot.

Be very careful that all transactions are accounted, workflows complete (as much as
possible), payments complete. Review the Best Practises from Oracle for more detailed
advice. If you don’t do this you will hit serious problems.

9. Post Go-Live. There are going to be problems on anything of this scale. Thats a given.
Have all teams on standby so that when issues arise, they can be addressed. With Oracle
Critical Support you’ll hopefully be in better shape than with standard support. As soon
as possible clone production to a test system and simulate the payroll, closing and other
key events in advance.

10. Party? If you do an R12 Upgrade, you deserve a big party. R12 is the most
challenging upgrade you will ever have done in Oracle ERP. Good Luck !!!

2 Responses to “R12 Upgrade Plan”

1. Vivek Says:
June 6, 2009 at 9:34 pm | Reply
You narrative gives a “what to expect” kind of inputs that would help any one
who is hesitant to go for R12.It is more of a project planning level cheat sheet. U
have often mentioned “Payables” & “Payment” modules to be the one that play
critical role in the overall success. Any plans to divulge more on those specific
area?

o irroberts Says:
June 7, 2009 at 3:43 am | Reply

Let me see if I can find some time to write on that in future. From what
we’ve seen so far (based on R12.0.4 RUP5 and a pile of other patches):

1. AP makes up 33% of all upgrade bugs (Payments actually on R12.0.4

RUP5 was not that bad)

2. We had to patch the QuickInvoice form (and still not fully working)

3. The Accruals (not strictly AP I know, but our AP users do this), does
not work. Expect patches from Oracle for this shortly. We’re working with
Development in the US and think it has been resolved. There are about
half a dozen bugs that we seem to be the first to come across.

4. Accounting on AP/PO with Exchange Rate Variances has problems for

us. Again Oracle has accepted this as a bug and issued a patch (although
the patch contained so much other stuff we’re not applying currently and
will stick with the custom config in XLA Module)

5. Prepayments – lots of bugs and lots of patches

6. Accounting generally was problematic (we applied another 4 patches to

fix another error very recently)

7. Critical Patch Sets – this is a balance and I’m sure there will be
disagreement, but you need to maintain a stable UAT environment and
keep the users on board and balance the need to apply these. We’ve opted
to apply some but will apply the remaining post upgrade as our UAT
seems largely OK now. The problem is if you apply these during UAT you
invalidate your UAT as they are so wide in terms of what they hit.
However if you don’t you also run the risk…….

8. Trial Balance doesn’t work – more patches.

9. If I was starting an R12 Upgrade, go for the latest release. It has all the
patch sets because the timeline of most R12 would mean you would have
time to resolve any critical issues, even if the release is new.
10. Watch out on Payments if you have lots of formats. They all need to
be rewritten. That’s a big job. We’re also heavily customized on Payments
so we;re probably more complex than most companies. Core payments has
been not bad on this release 12.0.4 RUP5