Upgrade failed due to issues with the wsbi servers and the different sites (Faculty, Provider and Kiosk) Rolled back at 12am, had to engage Mythics to assist with backing out of 19C oracle and putting it back to 12c then starting the roll back to the last good backup.

All complete by 3:57am. Tested and functional.

Meeting had with the APPS team to discuss what went wrong, what we learnt and how to move it forward asap.

Few lessons learnt, ensure we are more prepared, and Vendors fully understand our entire environment before starting. Go over the plan the day before the vendor in greater details than we did, involve as many of the team as possible for more input. Don’t allow vendors to make changes to the set up unless previously agreed upon. Be more detailed in our meetings that we cover every aspect of the entire set up.