Good reference for RAC/Grid

http://psoug.org/reference/rac.html

Step By Step process of Configuring Oracle 10gR2 (10.2.0.3) RAC Standby For RAC Primary with ASM and OMFs. http://www.oracledba.org/10g/dr/10gR2_dataguard_RAC_to_RAC.html#CREATE_PHYSICAL_STANDBY_Database:

Oracle® Real Application Clusters Administrator's Guide
http://www.stanford.edu/dept/itss/docs/oracle/10g/rac.101/b10765/toc.htm

DBCA failed at the end on 11gR2 grid

11.2 grid and 10g Oracle home. DBCA can create the database all the way to 100%. But after creating the database, it failed to start.

Solution:

Looking at Oracle Doc


Incorrect Ownership and Permission after Relinking or Patching 11gR2 Grid Infrastructure [ID 1083982.1]

Error when try to connect 11g ASM using 10g DBCA

11g R2 RAC environemnt and successfully installed 10.2.0.4. While runing dbca to create a database, got the following error

"DBCA could not startup the ASM instance configured on this node. To processd with database creation using ASM you need the ASM instance to be up and running. Do you want to recreate the ASM instance on this node?"

Solution, according to Oracle Note

Pre 11.2 Database Issues in 11gR2 Grid Infrastructure Environment [ID 948456.1], applied patch 8288940.

ORA-29701: unable to connect to Cluster Synchronization Service

11g R2 grid infrustructure on a standalone server. After the Server moved from one data center to another, go the following errors while start the ASM.

SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service

Solution:

run crsctl start resource -all.

[oracle@passl07 bin]$ ./crsctl start resource -all
CRS-2672: Attempting to start 'ora.LISTENER.lsnr' on 'passl07'
CRS-2672: Attempting to start 'ora.cssd' on 'passl07'
CRS-2679: Attempting to clean 'ora.diskmon' on 'passl07'
CRS-2681: Clean of 'ora.diskmon' on 'passl07' succeeded
CRS-2672: Attempting to start 'ora.diskmon' on 'passl07'
CRS-2676: Start of 'ora.diskmon' on 'passl07' succeeded
CRS-2676: Start of 'ora.cssd' on 'passl07' succeeded
CRS-5014: Agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" timed out starting process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" for action "start": details at "(:CLSN00009:)" in "/u01/app/oracle/product/11.2.0/grid/log/passl07/agent/ohasd/oraagent_oracle/oraagent_oracle.log"
CRS-5014: Agent "/u01/app/oracle/product/11.2.0/grid/bin/oraagent.bin" timed out starting process "/u01/app/oracle/product/11.2.0/grid/bin/lsnrctl" for action "check": details at "(:CLSN00009:)" in "/u01/app/oracle/product/11.2.0/grid/log/passl07/agent/ohasd/oraagent_oracle/oraagent_oracle.log"
(:CLSN00009:)Utils:execCmd aborted
CRS-2674: Start of 'ora.LISTENER.lsnr' on 'passl07' failed
CRS-2679: Attempting to clean 'ora.LISTENER.lsnr' on 'passl07'
CRS-2678: 'ora.LISTENER.lsnr' on 'passl07' has experienced an unrecoverable failure
CRS-0267: Human intervention required to resume its availability.
CRS-2672: Attempting to start 'ora.asm' on 'passl07'
CRS-2676: Start of 'ora.asm' on 'passl07' succeeded
CRS-2672: Attempting to start 'ora.PASS_DATA.dg' on 'passl07'
CRS-2672: Attempting to start 'ora.PASS_FLASH.dg' on 'passl07'
CRS-2672: Attempting to start 'ora.PASS_LOG.dg' on 'passl07'
CRS-2672: Attempting to start 'ora.PASS_BACKUP.dg' on 'passl07'
CRS-2676: Start of 'ora.PASS_FLASH.dg' on 'passl07' succeeded
CRS-2676: Start of 'ora.PASS_BACKUP.dg' on 'passl07' succeeded
CRS-2676: Start of 'ora.PASS_DATA.dg' on 'passl07' succeeded
CRS-2676: Start of 'ora.PASS_LOG.dg' on 'passl07' succeeded

ld: warning: libclntsh.so.9.0, needed by /psoft/psoftdev/hcmdev/bin/libpsora_ansi.so, not found (try using -rpath or -rpath-link)

New installation of PeopleSoft People Tools 8.49. Cobol compiled fine but got the following errors when linking:

[psoft@hcmdevdb01 setup]$ ./psrun.mak
./psrun.mak - linking PSRUN for redhat-4-ia32, Version 2.6.18-164.el5 ...
./psrun.mak - Error(s) encountered creating PSRUN!
./psrun.mak - See /psoft/psoftdev/hcmdev/setup/psrun.err for messages
[psoft@hcmdevdb01 setup]$ cat psrun.err
ld: warning: libclntsh.so.9.0, needed by /psoft/psoftdev/hcmdev/bin/libpsora_ansi.so, not found (try using -rpath or -rpath-link)
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIStmtExecute'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIStmtPrepare'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIInitialize'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCITransCommit'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIBindByName'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIAttrSet'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIDescriptorAlloc'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIStmtFetch'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIDescriptorFree'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIDefineByPos'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIHandleFree'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIStmtSetPieceInfo'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCISessionEnd'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIServerAttach'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIEnvInit'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIDateTimeGetDate'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIHandleAlloc'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIAttrGet'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIDateTimeGetTime'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIStmtGetPieceInfo'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIErrorGet'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCITransRollback'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCISessionBegin'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIServerVersion'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIServerDetach'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIDateTimeConstruct'
/psoft/psoftdev/hcmdev/bin/libpsora_ansi.so: undefined reference to `OCIParamGet

By searching in Google and Oracle support, all suggested to create a soft link. So I went to create the soft link under $ORACLE_HOME/lib as

ln -s /u01/app/oracle/product/10.2.0/db_1/lib32/libclntsh.so.10.1 libclntsh.so.9.0

But still it didn't work.

Finally, I figured out the problem is the permission. After applied the Oracle patch set, the permission on the Oracle home is tight. So I did the following:

cd $ORACLE_HOME/install
[oracle@hcmdevdb01 install]$ sh changePerm.sh

-------------------------------------------------------------------------------
Disclaimer: The purpose of this script is to relax permissions on some of the
files in the database Oracle Home so that all clients can access them.
Please note that Oracle Corporation recommends using the most restrictive file
permissions as possible for your given implementation. Running this script
should be done only after considering all security ramifications.
-------------------------------------------------------------------------------

Do you wish to continue (y/n) [n]: y
Spooling the error log /tmp/changePerm_err.log...
Finished running the script successfully
[oracle@hcmdevdb01 install]$

Then the Cobol link runs successfully.

Cloning an Oracle Home

I accidently installed Oracle on the wrong directory. So I decide to clone the home installed to the new location and delete the original one. Here is what I did:

1. Copy the home to the new location:

cp -rp product /u01/app/oracle

2. Go to the new home location, cd oui/bin and then run
./runInstaller -clone ORACLE_BASE="/u01/app/oracle" ORACLE_HOME="/u01/app/oracle/product/10.2.0/db_1" ORACLE_HOME_NAME="home1"

It asked to run root.sh as root.

Open database with inconsistent data

The backup script was accidentally deleted by SA. We don't have the backup and the customer require a recovery to a certain date. The only backup we have is an operating system backup and all archive logs. I started with a backup and applied archivelogs for about 2 days. But no matter how many archive logs applied, I still got the following message:

Specify log: {=suggested filename AUTO CANCEL}
cancel
Media recovery cancelled.
SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: 'E:\ORACLE\ORADATA\ASMP\SYSTEM01.DBF'

Solution:

at init file set _ALLOW_RESETLOGS_CORRUPTION=TRUE and then the database is opened.
Do a full export of the database, create a new one and do an import.

SQL> startup mount;ORACLE instance started.
Total System Global Area 453498908 bytesFixed Size 75804 bytesVariable Size 295604224 bytesDatabase Buffers 157286400 bytesRedo Buffers 532480 bytesDatabase mounted.SQL> recover database using backup controlfile until cancel;ORA-00279: change 754753809 generated at 08/24/2010 12:15:12 needed for thread1ORA-00289: suggestion : E:\ORACLE\ORADATA\ASMP\ARCHIVE\ARC82677.001ORA-00280: change 754753809 for thread 1 is in sequence #382677
Specify log: {=suggested filename AUTO CANCEL}
ORA-00279: change 754755854 generated at 08/24/2010 12:15:53 needed for thread1ORA-00289: suggestion : E:\ORACLE\ORADATA\ASMP\ARCHIVE\ARC82678.001ORA-00280: change 754755854 for thread 1 is in sequence #382678ORA-00278: log file 'E:\ORACLE\ORADATA\ASMP\ARCHIVE\ARC82677.001' no longerneeded for this recovery
Specify log: {=suggested filename AUTO CANCEL}
cancel
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistentORA-01110: data file 1: 'E:\ORACLE\ORADATA\ASMP\SYSTEM01.DBF'
ORA-01112: media recovery not started
SQL> alter database open resetlogs;
Database altered.
SQL>